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ONR  Award:  N00014-04-1-0825 
Henry  M.  Jackson  Foundation 

Nursing  Telehealth  Applications  Initiative 

Project  Objectives  and  Accomplishments  for  Award  Period:  August  17,  2005  -  August  16,  2006 
Period  of  Performance:  July  1,  2007  -  December  31,  2009 


Task  1)  Deliverable:  Development  and  testing/research  of  a  Cyber  Clinical  Environment 

(CCE),  encompassing  leading-edge  telehealth  and  information  technologies  for  application 

in  civilian  and  military  training  settings, 

GOALS: 

1)  To  develop  and  evaluate  the  use  of  a  cyber  clinical  education  environment  that  will  allow 
students  to  use  telehealth  and  information  technologies  to  gain  proficiency  in  practicing  in  a 
technology-enhanced  healthcare  system. 

2)  To  provide  opportunities  for  students  to  develop  skills  that  will  transfer  from  the  simulated 
environment  to  the  real  world  clinical  practice  arena. 


BACKGROUND  AND  PURPOSE: 

There  is  a  critical  need  for  the  nursing  profession  to  use,  evaluate,  and  recommend  best  practices  for  the 
application  of  technology  in  practice  and  education.  Various  telehealth  and  information  technologies 
offer  the  ability  to  manage  health  care  data  and  improve  access  to  health  care  information  (1).  Challenges 
for  educators  include  not  only  providing  technology-related  experiences  for  students,  but  also 
strategically  using  technology  to  improve  the  educational  process.  These  challenges  are  occurring  at  a 
time  when  nursing  and  allied  health  education  programs  are  attempting  to  respond  to  increased 
enrollments,  resulting  from  the  current  shortage  of  nurses  and  allied  health  professionals  in  numerous 
areas  of  our  country.  Many  students  are  being  turned  away  because  there  are  too  few  nursing  faculty  and 
available  clinical  placement  sites  to  provide  the  increasing  number  of  students  with  the  necessary  clinical 
experiences  (2,  3).  While  enrollments  in  all  levels  of  nursing  programs  increased  by  18%  in  2005,  the 
National  League  for  Nursing  estimates  that  147,465  qualified  student  applications  for  Nursing  Schools 
were  turned  away  due  to  the  shortage  of  faculty  (4).  An  additional  challenge  exists  in  rural  areas,  where 
limited  numbers  of  clinical  practice  areas  exist,  travel  to  and  from  these  locations  is  problematic,  and  the 
range  of  specialty  experiences  for  students  may  be  lacking. 
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The  timeliness  of  these  issues  in  nursing  education  is  emphasized  by  concomitant  changes  in  societal 
demographics.  The  number  of  elderly  patients  with  chronic  diseases  is  increasing.  Many  of  these 
individuals  remain  in  the  community,  in  either  home  or  assisted-living  settings,  often  with  limited  or 
insufficient  access  to  healthcare  services.  To  address  these  current  and  future  needs,  students  must  be 
provided  with  educational  experiences  that  allow  them  to  provide  nursing  care  for  this  demographic 
group.  However,  placing  students  in  community  settings  to  gain  these  practice  experiences  is  becoming 
increasingly  difficult  for  a  variety  of  reasons.  This  proposal  addresses  many  of  these  issues  by  using 
telehealth  technologies,  both  by  providing  students  with  these  necessary  educational  experiences,  while 
allowing  hands-on  experiences  with  remote  assessment  technologies  that  will  be  required  to  care  for 
community-based  patients  or  others  located  at  a  distance  from  the  provider  of  services. 

Telehealth  has  been  defined  as  “the  electronic  transfer  of  health  information  from  one  location  to  another 
for  purposes  of  health  promotion,  diagnosis,  consultation,  education,  and/or  therapy”  (5).  Telehealth 
encompasses  videoconferencing,  transmission  of  still  images,  e-health,  remote  monitoring,  nursing  call 
centers,  use  of  electronic  health  records,  and  continuing  health  care  professional  education.  Benefits  of 
the  use  of  telehealth  approaches  include  increased  access  to  care  and  more  efficient  use  of  resources  (6, 
7).  The  program  of  research  in  telehealth  at  Mount  Aloysius  College  thus  far  has  resulted  in  empirical 
support  for  the  benefits  of  this  mode  of  service  delivery  through  testing  and  evaluation  of  the  use  of 
telehealth  technologies  to  address  decreasing  resources  in  both  nursing  education  and  practice. 

Because  required  training  environments  are  often  not  available  in  the  traditional  format  to  schools  in  both 
urban  and  rural  environments,  this  proposal  seeks  to  address  the  problem  by  creating  a  simulated  clinical 
education  environment  in  which  students  learn  using  the  latest  telehealth  and  information  technology 
tools,  designed  for  the  dual  purpose  of  preparing  a  workforce  capable  of  meeting  the  demands  of  the 
practice  settings  of  the  future  and  meeting  the  healthcare  challenges  by  increasing  access  to  care. 

Since  2004,  the  Office  of  the  National  Coordinator  for  Health  Infonnation  Technology  has  been  guiding 
national  technology  initiatives  in  healthcare.  The  American  Health  Information  Community,  a  federal 
advisory  body,  is  charged  with  recommending  strategies  to  move  forward  the  development  of  Health 
Information  Technology  in  the  areas  of  consumer  empowerment,  chronic  care,  biosurveillance,  and 
electronic  health  records.  The  synergistic  relationship  between  telehealth  and  health  information 
technology  has  been  clearly  articulated  in  a  recent  issue  paper  published  by  the  American  Telemedicine 
Association  (ATA).  In  its  paper  entitled  “Telemedicine,  Telehealth,  and  Health  Information 
Technology,”  the  ATA  describes  the  importance  of  overcoming  resistance  to  the  adoption  of  technology 
in  healthcare.  In  the  issue  paper,  ATA  leaders  stated,  “accelerating  the  adoption  of  all  forms  of 
technology  in  healthcare  is  considered  critical  to  improved  outcomes,  expanded  access,  and  increased 
efficiency”  (8).  If  this  acceleration  is  to  occur,  nurses  and  other  allied  health  professionals  must  be 
educated  in  the  use  of  telehealth  and  health  information  technologies.  This  proposal  seeks  to  outline  best 
practices  related  to  training  nurses  and  allied  health  professionals  to  use  telehealth  technologies 
appropriately,  including  the  use  of  health  information  for  decision  support  ranging  from  individual 
patient-centered  to  community-centered. 


PLAN: 

The  development  and  evaluation  of  a  Cyber  Clinical  Environment  (CCE)  will  occur  in  parallel  on  three 
campuses:  Mount  Aloysius  College,  a  rural  college  in  Cresson,  PA;  Marquette  University,  an  urban 
university  in  Milwaukee,  WI;  and  the  Uniformed  Services  University  of  the  Health  Sciences  in  Bethesda, 
MD.  The  purpose  of  parallel  development  is  threefold:  1)  to  expand  the  pool  of  patient  encounters 
available  as  learning  experiences  for  students;  2)  to  allow  schools  to  share  faculty  expertise;  and  3)  to 
allow  students  from  the  three  campuses  to  interact  with  each  other  to  share  their  learning  experiences. 
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Phase  1)  Development  (Months  1-4)  -  Identify  learning  objectives,  case  studies,  equipment  sets,  learning 
outcomes,  patient  scenarios,  and  lesson  plans.  Develop  a  protocol  for  research  and  evaluation  of  the 
virtual  clinical  education  environment  and  obtain  1RB  approval.  It  is  envisioned  that  the  simulated 
clinical  education  environment  would  include  labs  in  which  students  would  use  remote  assessment 
equipment  housed  at  a  central  station  to  interview  and  examine  professional  patients  located  in  another 
area  of  the  lab,  thus  simulating  the  real-world  situation  of  patients  being  located  a  distance  from  the 
healthcare  provider.  In  the  military  setting,  students  would  use  telehealth  devices  developed  for 
deployment  operations  to  collect  and  transmit  clinical  data.  Vignettes  would  be  developed  for  use  with 
professional  patients,  providing  students  the  opportunity  to  develop  skills  in  health  assessment,  remote 
monitoring,  and  critical  thinking  to  support  decision-making.  In  addition,  relationships  will  be 
established  with  several  assisted- living  facilities  to  provide  a  venue  for  clinical  practice  in  remote, 
community-based  assessment  and  care  of  patients  with  chronic  diseases  such  as  diabetes  and 
hypertension.  Health  information  technology  strategies  could  be  incorporated  into  lab  activities  to 
provide  practice  with  information  management,  use  of  electronic  health  record,  and  data-based  decision 
support.  Variations  to  address  the  needs  of  graduate  education  at  the  military  site  would  include  research 
to  be  conducted  at  the  National  Capitol  Area  Simulation  (SIM)  Center  using  professional  patient  actors 
and  to  increase  the  realism  of  the  cases,  the  SIM  center  vignettes  will  be  based  on  actual  telehealth  cases 
collected  from  clinicians  serving  in  Operation  Iraqi  Freedom  and  Operation  Enduring  Freedom.  Finally, 
telehealth  technology  will  be  used  to  simulate  the  real-world  situation  of  patients  geographically 
separated  from  the  provider  during  Family  Nurse  Practitioner  (FNP)  and  Certified  Registered  Nurse 
Anesthesia  (CRNA)  clinical  rotations. 

Phase  2)  Implementation  and  pilot  testing  (Months  5-8)  -  Execute  protocol  and  conduct  preliminary 
evaluation  of  the  Cyber  Clinical  Environment  (CCE).  Every  effort  will  be  made  to  coordinate  clinical 
experiences  such  that  students  at  one  university  will  have  an  opportunity  to  conduct  interviews  and 
assessments  on  professional  patients  at  one  location  while  parallel  activities  are  occurring  at  another  site. 
Sites  then  could  switch  patients,  such  that  professional  patients  at  one  site  could  potentially  be  evaluated 
by  students  on  all  three  campuses.  This  sharing  of  professional  patients  is  an  important  consideration  in 
that  only  one  professional  patient  with  a  designated  condition  would  be  needed  to  provide  students,  from 
all  three  campuses,  an  opportunity  to  asses  and  interact  with  a  patient  experiencing  a  given  condition. 
Carefully  planning  these  experiences  could  increase  the  likelihood  that  all  students  graduating  from  these 
nursing  programs  would  have  a  supervised  opportunity  to  assess  and  interact  with  individuals  with 
common  chronic  health  problems.  Upon  completion  of  these  clinical  experiences,  one  faculty  member 
could  coordinate  a  discussion  with  students  from  one,  two,  or  all  three  campuses,  using 
videoconferencing  technologies. 


NT  A I  SIMULATION  WORKSHOP 

Thursday,  May  22  and  Friday,  May  23,  2008 

HJF  Telehealth  Department  at  Mount  Aloysius  College 

Attendees:  Telehealth  staff  which  includes  Project  Director,  Project  Coordinator,  Lab  Coordinator, 
Network  Engineer,  and  Administrative  Assistant,  MAC  Nursing  Instructor  for  health  assessment  and 
expert  trainer/presenter. 

The  completion  of  Room  322  clinical  lab  environment  has  resulted  in  the  exploration  of  all  the 
possibilities  that  this  setting  may  have  to  offer.  We  decided  to  schedule  a  2  day  workshop  that  would 
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include  some  of  these  possibilities  that  would  directly  enhance  the  nursing  learning  experience.  Our  goal 
was  to  explore  clinical  simulation  by  utilizing  the  modalities  we  have  access  to. 

Topics:  Simulation  mannequins  (high  fidelity  vs.  low  fidelity)  reviewed  usage  in  past 
studies;  pros  and  cons. 

Standardized  patients;  how  they  are  recruited,  trained,  and  used  in  clinical 
medical  training  environments,  advantages  and  versatility. 

Remote  assessment;  tools  and  equipment  that  are  incorporated  in  transferring 
assessment  information  over  distance.  What’s  out  there,  effectiveness,  pros  and  cons,  cost  effectiveness, 
etc. 


Videoconferencing;  equipment  available,  comparison  of  quality  and  cost, 
capabilities,  portability,  etc. 

Expert  advice  was  obtained  through  consultation  on  the  subjects  of  standardized  patients  and  on  content 
of  nursing  health  assessment  curriculum 

Objective:  We  wanted  to  explore  all  the  possibilities  of  furnishing  the  new  lab  to  create  optimal  usage  of 
the  setting  (operating  room,  emergency  room,  trauma,  1CU/CCU,  N1CU,  and  many  other  specialty 
settings)  while  staying  within  our  budget  confines. 

We  also  wanted  to  obtain  information  and  education  about  the  use  of  standardized  patients  in  the 
application  of  telehealth. 

Outcome:  Two  phases  of  the  Cyber  Clinical  Environment  study  will  utilize  standardized  patients.  This 
workshop  provided  valuable  information  to  help  us  recruit  and  train  standardized  patients  and  develop 
scenarios  for  this  study. 


Cyber  Clinical  Environment  Study 
Part  1 

A  pilot  study  of  the  CCE  was  implemented  using  20  RN  to  BSN  students  enrolled  in  Mount  Aloysius 
College  Nu  302  Health  Assessment  Class  from  July  2  through  July  23,  2008.  Standardized  patients  were 
used  in  the  assessment  scenarios  which  took  place  in  a  clinical  lab  setting  at  the  Greater  Altoona  Career 
and  Technology  Center  (GACTC).  Audio/video  conferencing  equipment  was  set  up  to  connect  the  lab 
setting  to  the  classroom  which  was  located  in  the  same  building,  but  on  a  different  floor.  Interaction 
between  the  students,  the  patient,  and  the  person  performing  the  assessment  took  place.  Remote 
assessment  tools  that  were  used  in  this  study  included  the  derma-scope  (skin  camera)  and  ampliphied 
stethoscope.  Assessment  scenarios  carried  out  for  the  study  included  skin,  respiratory,  and  cardiac.  The 
standardized  patients  filled  out  an  evaluation  on  the  student  doing  the  assessment.  Attitude/satisfaction 
surveys  were  filled  out  by  the  students  that  participated  in  the  study.  Study  activity  occurred  on  three 
different  class  nights  which  coincided  with  the  body  system  that  was  presented  in  the  health  assessment 
scenarios. 
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Figure  1.  Assessment  being  performed  on  a  standardized  patient. 


Figure  2.  Students  participating  in  remote  assessment  of  standardized  patient. 
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CYBER  CLINICAL  ENVIRONMENT  STUDY  UTILIZING 
STANDARIZED  PATIENT  FOR  NU302  HEALTH  ASSESSMENT 

July  2008 


Mount  Aloysius  College  conducted  the  Cyber  Clinical  Environment  research  to  evaluate  the  effectiveness 
of  new  technology  to  enhance  the  education  of  students  scheduled  for  Nursing  302  Health  Assessment 
class  as  participants.  Students  were  asked  to  interact  in  a  remote  health  assessment  process  that  involved 
standardized  patients  (people  that  acted  as  patients).  The  standardized  patient  and  the  nurse  performing 
the  assessment  were  videoconferenced  into  the  classroom  which  allowed  two-way  interaction  between  the 
class  and  nurse/patient. 

The  object  of  the  study  was  to  prove  that  a  simulated  clinical  education  environment  wrould  include  labs 
which  students  would  use  remote  assessment  equipment  housed  at  a  central  station  to  interview  and 
examine  professional  patients  located  in  another  area  of  the  lab,  thus  simulating  the  real-world  situation  of 
patients  being  located  a  distance  from  the  health  care  provider.  The  following  scenarios  were  presented: 

1)  skin  assessment  on  July  9,  2)  respiratory  assessment  on  July  16,  and  3)  cardiac  assessment  on  July  23. 
These  scenarios  were  developed  for  remote  health  assessment  and  critical  thinking  to  support  decision¬ 
making.  Each  class  period  lasted  approximately  40-45  minutes,  depending  on  the  number  of  students  and 
class  interaction. 

There  were  19/20  participants  who  returned  surveys  (10  males;  and  9  females).  The  groups  demographics 
indicated  that  students  40  years  of  age  and  older  was  (53%);  30-34  years  old  was  (32%);  18-25  years  old 
was  10%;  and  25-29  years  old  was  5%.  Regarding  RN  years  of  experience,  47%  had  1-5  years,  23%  had 
6-10  years,  12%  had  11-20  years,  and  only  6%  had  26-30  years.  RN  working  experience  included 
cardiology,  intensive  care,  emergency  medicine,  family  practice,  gastroenterology,  med/surg,  pediatrics, 
nursing  informatics,  operating  room,  and  telemetry.  At  the  conclusion  of  the  third  session  on  July  23,  2008, 
participants  in  this  study  were  asked  to  fill  out  a  survey  of  25  questions  in  sections  1-3  related  to  their 
Cyber  Clinical  Environment  experience.  Following  are  the  results  of  this  study: 

Quantitative  Data  Analysis:  (Male  and  Female  students  answered  all  questions  in  each  section  very 
similarly) 

Section  1  included  8  questions  related  to  the  students’  opinions  about  their  telehealth  experience  showed 
that  communicating  effectively  with  the  patient  was  rated  the  highest  out  of  8  questions  in  this  section, 
(53%  strongly  agreed  and  47%  agreed).  The  second  highest  (95%)  of  students  indicated  they  were  able  to 
utilize  the  remote  telepresenter  to  assist  with  the  assessment  effectively.  The  third  highest  (94%)  showed 
the  students’  believe  telehealth  technologies  allowed  them  to  make  correct  evaluations.  This  section  also 
showed  (84%)  of  students  were  favorable  on  telehealth  being  a  valuable  tool  for  nurses,  and  (79%)  could 
carry  out  assessments  without  physically  touching  the  patient.  Students  also  disagreed  that  telehealth  is 
tedious  (57.5%)  and  complicated  (68%).  The  majority  of  student’s  opinion  that  telehealth  could  save  time 
was  the  only  question  in  this  section  that  was  closely  rated  by  agreement  (37%)  and  disagreement  (42%). 

Section  2  included  1 1  questions  on  students’  assessment  abilities  to  communicate  via  videoconferencing 
showed  that 
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100%  of  students  agreed  to  strongly  agree  that  they  were  able  to  communicate  verbally  &  overall,  and 
believe  the  assistant’s  presenter  was  useful  to  them.  The  second  highest  (95%)  thought  they  were  looking 
down  upon  the  patient  due  to  camera  placement,  and  thought  the  lighting  during  the  consult  was  adequate. 
The  third  highest  (90%)  were  able  to  communicate  visually  as  needed.  This  section  also  showed  (83%)  of 
students  could  see  the  patient  well  in  regards  to  camera  angle  and  view.  Students  also  disagreed  (82%)  on 
their  belief  that  telehealth  technologies  products  decreased  their  ability  to  function  effectively  with  the 
patients,  (79%)  disagreed  they  were  frustrated  when  giving  directions  to  the  patient,  and  (68%)  disagreed 
they  felt  frustrated  when  trying  to  complete  an  assessment  using  telehealth  technologies.  Students  showed 
diversity  between  agreeing  (57%)  and  disagreeing  (42%)  on  their  ability  to  make  eye  contact  with  the 
patient. 

Section  3  included  6  questions  on  overall  satisfaction  with  students’  experience  showed  that  (94%)  agreed 
they  could  envision  potential  applications  for  using  telehealth;  (85%)  would  like  to  have  more 
opportunities  to  learn  about  telehealth;  and  (83%)  would  consider  using  telehealth  technologies  in  their 
nursing  practice.  Students  also  disagree  ((90%)  that  they  are  not  interested  in  telehealth  technologies,  and 
(89%)  disagree  that  telehealth  applications  represent  unnecessary  health-care  expenditures.  One  final 
result  showed  (89%)  of  students  agreed  they  prefer  a  face-face  consultation  over  telehealth. 

SUMMARY: 

This  study  of  NUR302  students  documents  the  potential  uses  and  the  values  of  this  methodology  in 
the  learning  of  RN  to  BSN  baccalaureate  program. 


Part  2 

Another  planned  activity  for  students  in  the  undergraduate  programs  includes  placing  telemonitoring 
kiosks  in  assisted  living  facilities.  These  devices  would  be  equipped  with  daily  monitoring  devices  that 
can  be  individually  configured  for  specific  patient  needs.  Monitoring  devices  might  include  weight 
scales,  blood  pressure  cuffs,  heart  rate  sensors,  pulse  oximeters,  glucometers,  dermatology  cameras, 
wound  cameras,  ophthalmoscopes,  etc.  In  addition,  daily  condition-specific  questions  can  be  asked  and 
responses  monitored.  There  are  formalized  teaching  programs  already  programmed  into  these  kiosks  that 
students  can  review  with  the  patients.  Videoconferencing  capabilities  are  available  as  well.  Each  site  will 
provide  an  opportunity  for  8-10  undergraduate  nursing  students  to  monitor  and  interact  with  one 
individual  in  an  assisted  living  facility,  over  the  course  of  a  semester.  This  type  of  experience  at  the 
undergraduate  level  will  assist  students  in  preparing  to  care  for  community  dwelling  individuals  with 
chronic  health  conditions,  provide  them  with  opportunities  to  demonstrate  and  refine  critical  thinking 
skills,  and  develop  a  level  of  comfort  and  proficiency  with  providing  nursing  care  remotely. 


Phase  3)  Testing,  iterative  development,  and  evaluation  (Months  9  -12)  -  Modify  approach  based  on 
findings  from  Phase  2;  final  testing,  evaluation,  and  report  writing.  Upon  completion  of  the  remote 
clinical  experiences,  students  will  complete  evaluation  forms,  indicating  their  comfort  with  the 
technology,  their  perceived  ability  to  carry  out  a  comprehensive  assessment  remotely,  their  perceived 
ability  to  communicate  effectively  and  professionally  with  the  patient,  as  well  as  the  specific  positive  and 
negative  aspects  of  the  experience.  In  particular,  evaluations  will  explore  aspects  of  the  experience  that 
students  feel  should  be  modified  to  enhance  the  educational  experience.  In  addition  to  evaluating  the 
student  performance  during  the  encounter,  SP’s  will  be  asked  to  provide  feedback  about  the  encounter 
from  the  patient  perspective.  This  information  will  be  used  to  further  refine  the  protocol.  As  these 
individuals  have  undergone  many  in-person  physical  examination  procedures  with  students,  we  will 
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attempt  to  gather  information  about  their  perception  of  the  experience,  with  particular  emphasis  on  what 
aspects  could  be  improved,  and  if  they  believed  the  assessment  was  comparable  to  an  in-person 
evaluation.  Again,  we  will  use  this  information  to  refine  the  protocol.  Because  some  faculty,  particularly 
at  the  undergraduate  sites,  may  have  had  little  experience  with  these  technologies,  it  will  be  important  to 
gather  assessment  data  from  these  individuals  as  well.  In  addition  to  the  types  of  information  indicated 
for  students  and  professional  patients,  we  will  gather  information  about  the  faculty  perceptions  of  the 
learning  experience  for  the  students,  the  quality  of  the  encounters,  the  availability  and  quality  of  needed 
equipment  and  technology,  and  any  suggestions  for  further  refinement.  Evaluations  will  be  used  to  revise 
protocols;  they  will  be  included  in  the  final  report  and  will  be  written  up  for  publication.  Data  emerging 
from  this  project  will  support  the  value  of  telehealth  technology  to  prepare  students  to  meet  the  challenges 
of  21st  century  healthcare. 


The  other  activity  planned  under  this  initiative  specifically  for  undergraduate  nursing  students,  the 
placement  of  kiosks  in  assisted  living  facilities,  will  be  evaluated  in  much  the  same  way.  However,  this 
experience  will  take  place  over  the  entire  semester,  and  students  and  actual  patients  will  be  involved. 
Data  will  be  gathered  from  the  students,  patients,  and  faculty  members.  One  nuance  to  this  experience 
that  we  will  be  able  to  explore  is  the  ability  of  the  students  and  patients  to  set  up  and  keep  appointments, 
the  ability  of  the  students  to  follow  up  in  a  timely  manner,  if  indicated,  and  the  effectiveness  of  using 
formalized  teaching  programs  to  assist  in  disease  management.  As  noted  in  the  previous  example,  these 
data  will  be  used  to  disseminate  the  value  of  this  type  of  educational  experience  for  nursing  students,  in 
order  to  prepare  them  to  meet  the  ever  expanding  demands  to  provide  quality  healthcare  for  all.  Again, 
use  of  this  type  of  approach  will  help  to  bridge  the  barriers  of  distance  and  time,  while  addressing 
challenges  encountered  with  the  nursing  faculty  and  practicing  nurse  shortages. 


Parts  of  the  Cyber  Clinical  Environment  Study  took  place  on  two  other  campuses;  USUHS  and 
Marquette  University.  Following  are  the  two  reports  from  these  sites  on  their  studies. 

Report  from  USUHS  (site)  on  “Harnessing  Technology  to  Teach  a  Dying  Art:  Using  Telehealth 
Technology  to  Teach  Physical  Exam  Skills”  which  was  part  of  the  Cyber  Clinical  Environment 
study. 

Project  Number:  G261HP 
Principal  Investigator:  Dianne  Seibert 
Department:  GSN 


Final  Report 

Period  covered  by  this  report:  from  January  2007  to  December  2009 
Publications,  Abstracts,  and  Presentations: 

List  all  manuscripts  submitted  for  publication  during  the  period  covered  by  this  report  resulting  from  this 
project.  Include  those  in  the  categories  of  lay  press,  peer-reviewed  scientific  journals  invited  articles,  and 
abstracts.  Each  entry  must  include  the  author(s),  article  title,  journal  [book,  editor(s),  publisher,  volume 
number,  page  number(s),  and  date 


9 


(1)  Lay  Press:  None 

(2)  Peer-Reviewed  Scientific  Journals:  None 

(3)  Invited  Articles:  None 

(4)  Abstracts:  None 


Summarize  the  progress  during  the  period  of  this  report  and  its  impact  on  your  plans  for  the 
remainder  of  the  project. 

This  project  was  a  sub-project  under  a  larger  Telehealth  Grant  spearheaded  by  Dr  Janet  Grady  at  Mount 
Aloysius  College.  The  parent  grant  was  to  look  at  the  use  of  Telehealth  tools  in  nursing  education 
programs,  and  Dr  Grady  was  interested  in  partnering  with  the  Family  Nurse  Practitioner  students  in  the 
Graduate  School  of  Nursing  at  USU.  Her  undergraduate  students  were  going  to  use  telehealth  technology 
to  remotely  evaluate  patients  in  a  nursing  home  setting.  The  particular  tool  and  patient  population  she 
planned  to  use  was  not  appropriate  for  students  in  a  Nurse  Practitioner  (NP)  program,  however,  so  this 
sub-study  was  designed  to  meet  the  learning  needs  of  NP  students. 

Due  to  a  prolonged  continuing  resolution,  it  was  over  a  year  before  the  Office  of  Naval  Research  (funding 
agency)  through  the  Henry  Jackson  Foundation,  released  the  funds  to  begin  the  study.  During  those 
intervening  months,  new  telehealth  technologies  emerged,  and  the  study  focus  shifted  dramatically.  This 
specific  protocol  was  never  put  into  place  -  instead,  two  other  studies,  centering  on  the  newly  developed 
telehealth  tools,  were  designed.  Both  of  the  studies  were  built  within  the  same  budget,  and  they  each 
have  their  own  project  numbers. 


Summarize  the  progress  during  the  period  of  this  report  toward  the  achiev  ement  of  the  originally 
stated  aims  and  list  the  significant  results: 

This  study  was  completed  by  8  physician  providers  and  36  students  (School  of  Medicine  and  Graduate 
School  of  Nursing).  Providers  commented  that  the  tool  was  useful,  but  would  have  been  more  useful  if 
the  images  recorded  had  colors  that  were  more  lifelike.  They  also  commented  that  the  “learning  curve” 
for  using  the  hand  pieces  was  step.  The  students  appeared  to  appreciate  the  tools  for  learning  difficult 
physical  exam  skills,  but  perhaps  more  importantly,  commented  that  having  access  to  clinical  experts  at  a 
point  in  their  career  where  this  expertise  could  be  appropriately  appreciates  was  very  valuable.  Access  to 
experts  when  they  were  very  novice  learners  was  not  as  helpful  as  access  to  experts  later  in  their 
educational  programs  (as  this  interaction  was).  Data  was  collected  anonymously;  none  of  the  subjects 
provided  any  identifying  information  on  the  survey  instruments.  The  GSN  faculty  &  student  encounters 
were  video  recorded  for  viewing  by  the  remote  nursing  students  (part  of  the  study  design),  but  these 
recordings  were  not  used  in  data  collection  or  analysis. 


In  layperson’s  terms,  summarize  the  progress  during  the  period  of  this  report: 

This  study  showed  that  the  use  of  telehealth  tools  to  teach  health  assessment  skills  -  even  to  relatively 
seasoned  clinicians  -  is  very  useful.  It  was  difficult  to  determine  how  much  impact  the  tools  themselves 
had  on  the  outcome,  because  just  having  access  to  expert  clinicians  appeared  to  be  helpful  in-and-of  itself. 

In  layperson’s  terms,  explain  any  medical/military  significance  or  implications  of  your  results  to 
date: 
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This  has  already  had  some  impact  on  the  educational  process  within  the  GSN.  During  the  2nd  year  of  the 
program  (practicum  courses)  the  faculty  is  setting  up  similar  instructional  sessions  with  expert  clinicians 
to  teach  advanced  skills  using  these  telehealth  tools. 


Progress  Report 
Jill  M.  Winters,  PhD,  RN 
Marquette  University  Site 


Goals  of  the  project: 


1)  To  develop  and  evaluate  the  use  of  a  clinical  education  environment  that  will  allow  students  to  use 
telehealth  and  information  technologies  to  gain  proficiency  in  practicing  in  a  technology-enhanced 
healthcare  system. 

2)  To  provide  opportunities  for  students  to  develop  skills  that  will  transfer  from  the  simulated  environment 
to  the  real  world  clinical  practice  arena. 

In  order  to  prepare  for  the  student  experience,  telehealth  equipment  was  evaluated  to  determine  the  most 
appropriate  equipment  to  purchase  for  optimal  student  experiences.  Telehealth  laboratories  were 
configured.  Equipment  was  purchased  and  installed.  Site  PI  and  student  research  assistants  were  trained  to 
use  the  equipment. 

Cases  were  developed  for  simulated  assessment  experiences  for  the  undergraduate  students  in  the 
assessment  course.  Along  with  these  cases,  evaluation  forms  were  developed,  provided  standardized 
patients  with  a  structure  for  providing  students  with  feedback.  Standardized  patients  were  hired  and 
trained.  Fifty  five  subjects  (students)  have  completed  two  of  the  assessment  experiences  to  date.  Four 
assessment  experiences  are  planned  for  the  summer  students.  Preliminary  feedback  from  both  students 
and  standardized  patients  has  been  very  positive. 

For  the  research  portion  of  this  project,  IRB  approval  was  obtained.  Upon  completion  of  the  four 
assessment  experiences,  students  will  provide  data  about  their  comfort  with  equipment,  ability  to  carry 
out  remote  assessment,  comparisons  of  assessment  activities  face-to-face  and  remotely. 


The  last  phase  (Phase  III)  of  the  Cyber  Clinical  Environment  study  involved  placing  multi-user  kiosks 
in  two  different  assisted  living/managed  care  facilities  and  one  senior  citizen  center. 

Phase  III  planned  activity  included  placing  telemonitoring  kiosks  in  assisted  living  facilities.  These 
devices  can  be  equipped  with  daily  monitoring  devices  individually  configured  for  specific  patient  needs. 
Monitoring  devices  capability  includes  weight  scales,  blood  pressure  cuffs,  heart  rate  sensors,  pulse 
oximeters,  glucometers,  dermatology  cameras,  wound  cameras,  ophthalmoscopes,  etc.  In  addition,  daily 
condition-specific  questions  can  be  asked  and  responses  monitored,  and  formalized  patient  teaching 
programs  can  be  implemented  and  evaluated.  Remote  assessment  equipment  was  purchased  from 
VitelNet.  3  multi-user  kiosks  (computers)  were  placed  in  three  different  sites;  a  managed/assisted  living 
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community,  a  managed  apartment  complex,  and  a  senior  citizen  center.  37  participants  with  one  or  more 
of  the  following  chronic  diseases,  diabetes,  COPD,  hypertension,  and  CHF,  met  the  criteria  for  the  study 
and  were  consented  and  trained  on  the  use  of  the  remote  assessment  equipment.  The  kiosks  were 
connected  to  a  database  at  the  college  and  managed  by  the  project  coordinator  in  making  live  video 
connections  once  a  week  and  monitoring  their  vital  signs  that  were  taken  as  often  as  the  participant 
desired.  Questions  were  also  asked  related  to  their  chronic  condition(s)  in  which  they  answered  on  the 
computer.  Education  was  provided  about  better  self-management  of  their  condition.  This  study  was 
conducted  over  six  months;  June,  2009  through  November,  2009. 


Concise  Accomplishments: 

Video  connections  were  done  every  Tuesday,  Wednesday,  and  Thursday  to  participants  as  part  of  the 
weekly  remote  assessment  and  education  related  to  their  chronic  disease.  Each  participant  was  assigned  a 
day  and  time  to  do  their  video  connection.  The  video  connections  were  scheduled  30  minutes  apart,  which 
allowed  some  extra  time  for  participants  to  make  the  connection  and  take  and  transmit  their  vital  signs 
before  speaking  with  the  Telehealth  nurse.  Vital  signs  that  were  entered  into  the  computer/database 
included  blood  pressure,  pulse  oximetry,  weight,  and  pulse.  Participants  with  diabetes  were  asked  to  enter 
their  glucometer  readings.  Reports  of  the  participants’  vital  signs  were  collected  into  the  database  which 
were  printed  out  and  sent  to  the  participants  to  allow  them  to  share  with  their  primary  care  doctors  and 
family  members.  The  live  connections  started  the  first  of  June  and  continued  through  November,  2009. 
Questionnaires  about  their  chronic  condition  were  given  to  each  participant  before  and  after  the  study  and 
results  were  compared  to  assess  any  increase  in  knowledge.  Self-efficacy  and  satisfaction  surveys  were 
given  at  the  end  of  the  study  to  evaluate  the  participant’s  opinions  about  the  study  and  the  technology 
used  in  the  study.  Thirty  seven  people  started  the  study  and  29  completed  the  study.  The  equipment  set 
used  in  the  study  presented  some  technical  challenges  that  were  not  clarified  by  the  vendor  prior  to  the 
study’s  implementation.  After  the  study  was  underway,  the  vendor  made  it  known  that  the  multi-user 
kiosk  with  video  had  not  previously  been  used  in  a  research  application,  and  this  situation  resulted  in 
unexpected  beta  testing  of  the  equipment. 


Work  Plan: 

Our  plan  for  Phase  Ill  of  the  Cyber  Clinical  Environment  study  was  to  finalize  data  analysis.  This  project 
closed  December,  2009.  All  data  have  been  collected,  and  analyzed.  One  of  our  objectives  for  this  phase 
of  the  study  was  to  develop  and  evaluate  the  use  of  a  cyber  clinical  education  environment  that  will  allow 
students  to  use  telehealth  and  information  technologies  to  gain  proficiency  in  practicing  in  a  technology- 
enhanced  healthcare  system.  We  were  unable  to  meet  the  objective  of  student  involvement,  mainly  due  to 
the  timing  of  the  study  implementation  which  occurred  at  the  end  of  the  spring  semester.  Because  of  the 
academic  schedule,  nursing  students  were  not  able  to  commit  to  a  six  month  study  that  ran  through  the 
summer  months. 

Another  objective  was  to  provide  a  means  of  remote  assessment  that  afforded  the  participants  the 
opportunity  to  learn  more  about  their  chronic  disease  (diabetes,  COPD,  CHF,  and  hypertension)  and  to  be 
better  prepared  to  manage  it.  Data  was  gathered  from  the  residents  and  telehealth  staff  to  evaluate  the 
learning  experience  and  the  value  of  the  telemonitoring  experience  for  the  residents.  Nuances  to  this 
experience  that  we  are  able  to  explore  include  the  willingness  of  residents  to  set  up  and  keep 
appointments,  the  effectiveness  of  using  formalized  teaching  programs  to  assist  in  chronic  disease 
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management,  and  any  association  between  comfort  and  acceptance  of  technology  and  improved  health 
outcomes  and  perceived  quality  of  life. 

Major  Problems/Issues: 

Equipment  was  purchased  and  training  was  done  with  the  telehealth  staff.  The  multiuser  kiosks  were 
placed  in  two  assisted  living  facilities  and  a  senior  citizen  center  which  all  required  internet  connection 
which  didn’t  previously  exist.  Initially,  we  encountered  problems  with  the  live  video  connections  and 
with  information  coming  into  the  database.  About  a  month  into  the  study,  we  were  made  aware  that  we 
were  the  first  to  use  the  multiuser  kiosk  with  live  video  capabilities.  As  a  result,  much  beta  testing  for  the 
vendor  occurred.  Some  data  we  had  intended  to  collect  are  missing  due  to  problems  with  the  equipment 
coupled  with  the  fact  that  the  internet  database  was  not  licensed  or  available  to  us  until  the  third  month  of 
the  study.  We  also  experienced  some  participant  attrition. 

Technology  transfer: 

While  no  direct  technology  transfer  has  occurred  to  date,  analysis  from  this  study  will  be  beneficial  in 
planning  and  implementing  future  studies  involving  telehealth  technology  and  chronic  disease 
management  for  patients  in  rural  community  settings.  As  noted  previously,  results  will  be  used  to 
disseminate  the  value  of  this  type  of  experience  to  bridge  the  barriers  of  distance  and  time,  while 
addressing  challenges  encountered  with  the  nursing  shortage,  particularly  as  it  affects  rural  areas.  From  a 
human  factor’s  standpoint,  the  importance  of  participants’  attitude  toward  technology  and  its  impact  on 
program  effectiveness  is  essential  information  for  anyone  planning  to  replicate  this  study  or  similar 
applications. 


Cyber  Clinical  Environment  study  participant  taking  his  vital  signs  and  participating  in  live  video 
connection  with  Telehealth  personnel  at  Mount  Aloysius  College. 
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Cyber  Clinical  Environment  Study 


Introduction 


There  is  considerable  interest  in  the  health  care  community  in  developing  new  strategies  that  are  effective 
in  enhancing  the  assimilation  and  retention  of  information,  and  the  development  of  skills,  necessary  for 
patient  self-management  of  chronic  disease  such  as  diabetes,  hypertension,  chronic  obstructive  pulmonary 
disease,  and  congestive  heart  failure.  People  with  chronic  disease  are  expected  to  follow  treatment 
regimens  that  require  a  significant  amount  of  knowledge  and  multiple  skills.  Educational  programs 
established  to  target  patient  self-care  are  essential  for  these  individuals.  Frequent  and  ongoing 
reinforcement  of  this  education  can  enhance  motivation  and  self-efficacy  and  may  be  the  most  valuable 
part  of  the  education  process. 


There  are  many  barriers  between  the  patient  and  the  education  including  the  shortage  of  nurses,  shorter 
hospital  stays,  limited  access  to  care  in  rural  areas,  availability  of  current,  evidenced  based  education,  and 
limited  income/insurance.  The  timeliness  of  these  issues  is  emphasized  by  concomitant  changes  in 
societal  demographics.  The  number  of  elderly  patients  with  chronic  diseases  is  increasing.  Many  of  these 
individuals  remain  in  the  community,  in  either  home  or  assisted-living  settings,  often  with  limited  or 
insufficient  access  to  healthcare  services. 


In  addition  to  face  to  face  education  provided  by  health  professionals,  various  telehealth  and  information 
technologies  offer  the  ability  to  manage  health  care  data  and  improve  access  to  health  care  information 
(Simpson,  2005).  Telehealth  encompasses  videoconferencing,  transmission  of  still  images,  e-health, 
remote  monitoring,  nursing  call  centers,  use  of  electronic  health  records,  and  continuing  health  care 
professional  education.  The  use  of  telehealth  technologies  would  be  especially  useful  in  rural  areas, 
where  limited  numbers  of  clinical  practice  areas  exist,  travel  to  and  from  these  locations  is  problematic. 


Benefits  of  the  use  of  telehealth  approaches  include  increased  access  to  care  and  more  efficient  use  of 
resources  (Dimmick,  Mustaleski,  Burgiss,  &  Welsh,  2000;  Peck,  2005).  Increased  availability  of 
telehealth  technology  can  assist  patients  to  achieve  better  management  of  their  ailments  thus  improv  ing 
self-efficacy.  Self  -  efficacy,  a  concept  from  social  cognitive  theory,  has  been  shown  to  be  related  to 
many  positive  health  behaviors  (Hogenmiller,  Atwood,  Lindsey,  Johnson,  Hertzog,  and  Scott,  2007).  The 
primary  architect  of  self-efficacy,  Bandura  (1994),  defines  the  concepts  as  peoples  “capabilities  to 
produce  designated  levels  of  performance  that  exercise  influence  over  events  that  affect  their  lives”  (p  1). 
Thus,  self-efficacy  is  expected  to  promote  successful  health  habits  such  as  self  monitoring  and  self¬ 
regulation  (Bandura,  2005).  Moreover,  it  is  through  mastery  experiences  that  self-efficacy  is  enhanced. 
Improved  self-care,  along  with  increments  in  self-efficacy,  may  be  expected  to  increase  patients’ 
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perceptions  about  their  quality  oflife.  Positive  experiences  with  the  telehealth  equipment  should  also 
make  their  attitudes  about  technology  more  positive  and  increase  their  confidence  in  the  use  of  computer 
and  televideo  equipment. 


This  pilot  study  evaluated  the  effectiveness  of  weekly  remote  assessment  of  residents  in  assisted  living 
and  managed  care  facilities  with  chronic  diseases  (or  conditions).  The  study  was  designed  to  explore  the 
following  questions: 


•  How  does  telemonitoring  impact  participants’  health  indices  (e.g.,  blood  pressure,  heart  rate)? 

•  How  satisfied  is  the  participant  with  the  remote  assessment/educational  process/ 

•  Are  there  changes  participant’s  perceived  sense  of  self  efficacy  in  controlling  his/her  disease(s). 

•  Is  there  a  change  in  the  participant’s  knowledge  about  his/her  chronic  disease? 

•  Are  the  participants  willing  to  accept  and  use  the  telehealth  technology  for  management  of  their 
chronic  disease? 

Method 

Telemonitoring  kiosks  with  videoconferencing  capabilities  were  placed  in  assisted  living  and/or  managed 
facilities  (apartment  complexes  or  senior  citizen  centers).  These  devices  were  equipped  with  monitoring 
devices  that  were  individually  configured  for  specific  patient  needs.  Monitoring  devices  included  one  or 
several  of  the  following,  depending  on  assessment  needs:  weight  scales,  blood  pressure  cuffs,  heart  rate 
sensors,  and  pulse  oximeters.  The  kiosks  also  contained  teaching  materials  targeted  to  the  participants’ 
specific  learning  needs.  In  addition  to  measurements  from  these  devices,  health  providers  were  able  to 
ask  condition-specific  questions  with  the  participant’s  responses  monitored. 


A  telehealth  registered  nurse  followed  participants  for  a  total  of  six  months.  The  nurse  monitored  all  data 
collected  through  the  VitelNet  database.  Participants  were  encouraged  to  enter  their  vital  signs  daily, 
Monday  through  Thursday  if  possible,  but  at  least  once  a  week.  The  database  was  checked  at  least  once  a 
day,  Monday  through  Thursday  by  telehealth  research  staff.  Participants  were  advised  that  the  VitelNet 
equipment  was  not  to  be  used  as  a  substitute  for  obtaining  medical  attention. 


Live  televideo  connection  with  a  telehealth  registered  nurse  was  scheduled  once  a  week  on  a  mutually 
agreed  upon  day  and  time.  During  the  weekly  televideo  conferences,  the  nurse  responded  to  the 
participants’  individual  concerns  and  questions,  and  reinforced  informational  content  provided  in  the 
teaching  materials.  In  the  event  that  data  being  entered  by  a  participant  was  abnormal,  the  participant  was 
advised  by  the  telehealth  registered  nurse  to  report  these  results  to  his  or  her  physician.  If  the 
abnormalities  were  considered  to  put  the  individual  at  risk  for  an  event,  the  participant  as  well  as  the 
manager/director  of  the  facility  was  notified.  Notifications  as  well  as  follow  ups  were  documented. 
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Data  was  gathered  from  the  participants  and  telehealth  staff  to  evaluate  the  learning  experience  and  the 
value  of  the  telemonitoring  experience  for  the  residents.  Nuances  to  this  experience  that  we  explored 
include  the  willingness  of  participants  to  set  up  and  keep  appointments,  the  effectiveness  of  a  formalized 
teaching  programs  to  assist  in  chronic  disease  management,  and  any  association  between  acceptance  of 
technology  and  improved  health  outcomes  and  attitudinal  measures  such  as  self-efficacy  and  perceived 
quality  of  life. 


The  participants’  attitudinal  data  were  obtained  via  self-report  questionnaires.  Five  short  questionnaires 
were  constructed:  comfort/competence  with  technology,  self  efficacy,  quality  of  life,  acceptance  of 
telemedicine  equipment,  and  satisfaction  with  current  life  situation.  In  each  case  items  were  written  to 
have  face  validity  and  the  reliability  on  each  questionnaire  was  assessed  in  terms  of  Coefficient  Alpha 
(Cronbach,  1951).  From  Table  1,  show  ing  the  number  of  items  and  the  Coefficient  Alpha  for  each  scale, 
we  can  see  that  four  out  of  the  five  questionnaires  have  high  reliability  and  one  demonstrated  moderately 
high  reliability. 


Table  1.  The  Number  of  Items  and  Reliability  for  Each  of  the  Five  Attitudinal  Questionnaires 


Scale 

No.  of 

Items 

Coef 

Alpha 

Quality  of  Life 

6 

0.85 

Satisfaction  with  Current  Life  Situation 

7 

0.92 

Self  Efficacy 

18 

0.73 

Comfort/Competency  with  Technology 

9 

0.94 

Acceptance  of  Telemedi  Equipment 

6 

0.91 

Results 


Thirty  seven  participants  started  in  the  program;  of  these  29  completed  the  six-month  program.  The 
participants  were  distributed  across  three  facilities. 


We  analyzed  the  participants’  pre-  and  post  scores  on  the  knowledge  test  about  their  chronic  disease  and 
on  four  of  the  self-report  questionnaires:  1)  comfort/competence  with  technology,  2)  self  efficacy,  3) 
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quality  of  life,  and  4)  satisfaction  with  current  life  situation.  We  hypothesized  that  participation  in  the 
program  would  increase  the  participants’  scores  on  each  of  these  variables.  We  found  a  significant 
improvement  in  the  participants’  self-efficacy  scores  (t(25)  =  2.98,  p<.03,  one  tail;  see  Figure  1).  None  of 
the  other  attitudinal  measures  or  the  knowledge  measure  changed  significantly  from  the  pre  to  the  post 
test.  We  also  analyzed  the  participants’  vital  sign  data  taken  over  the  course  of  the  study,  but  found  no 
significant  patterns  of  change  over  time. 


Figure  1.  Change  in  Self  Efficacy  from  pre  to  post  study  measurement 


At  the  end  of  the  study  we  administered  a  fifth  questionnaire  that  assessed  the  participants’ 
acceptance/satisfaction  of  telemedicine  equipment  they  used  during  the  study.  We  dichotomized  the 
scores  at  the  midpoint  (3.92  on  a  5-point  scale)  and,  based  on  their  scores,  classified  participants  as  either 
high  or  low'  on  acceptance  of  the  telemedicine  equipment.  We  used  this  as  an  independent  variable  to 
analyze  the  post-study  knowledge  test  and  the  four  questionnaire  scores.  We  hypothesized  that  those  wiio 
were  more  favorable  towards  the  telemedicine  equipment  would  have  higher  scores  than  those  who  were 
less  favorable. 


We  found  that  those  who  were  more  favorable  toward  the  equipment  had  significantly  higher  scores  on 
the  post-test  knowledge  test  (t(26)=  1.82,  p<..05,  one  tail;  see  Figure  2).  As  one  would  expect,  the 
analysis  also  showed  that  those  who  were  more  satisfied  with  the  telemedicine  equipment  had 
significantly  higher  scores  on  their  comfort  with  technology  (F(l,  25)  =  6.46,  p  <  .02;  see  Figure  3.) 
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Low  High 

Satisfaction  With  Telemedicine  Equipment 


Figure  2.  Post  test  knowledge  scores  for  those  low  and  high  on  satisfaction  with  telemedicine  equipment 


Figure  3.  Post-test  measure  of  comfort  with  technology  for  those  low  and  high  in  satisfaction  with  the 
telemedicine  equipment 


To  further  explore  the  impact  of  comfort  with  technology,  we  conducted  a  within-between  analysis  of 
variance  (ANOVA)  with  time  (pre,  post)  as  the  within  subjects  variable  and  satisfaction  with  the 
telemedicine  equipment  (low,  high)  as  the  between  subjects  variable.  We  hypothesized  that  there  would 
be  a  significant  interaction  tenn  -  specifically  that  the  post-test  scores  for  those  who  were  more  positively 
disposed  toward  the  equipment  would  be  higher  than  their  pre-test  scores  whereas  the  post  test  scores  for 
those  less  positively  disposed  to  the  technology  would  stay  constant  or  even  decrease.  We  found  a 
significant  interaction  on  three  of  the  dependent  variables:  comfort  with  the  technology  (F(l ,  25)  =  4.38, 
p  <  05),  quality  of  life  (F(l,  23)  =  1 1.91,  p  <  .003),  and  satisfaction  with  current  life  situation  (F(l,  24)  = 
9.03,  p  <  .007)  (see  Figures  4,  5,  and  6,  respectively).  Consistent  with  our  hypothesis,  those  who  were 
more  satisfied  with  the  technology  tended  to  increase  on  these  variables  from  the  beginning  to  the  end  of 
the  study  whereas  those  who  were  less  satisfied  with  the  technology  decreased  from  the  pre-  to  the  post¬ 
study  assessment. 
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Figure  4.  Pre-  and  post-study  measures  of  comfort  with  technology  for  those  low  and  high  in  satisfaction 
with  the  telemedicine  equipment 


Figure  5.  Pre-  and  post-study  score  on  quality  of  life  for  those  low  and  high  in  satisfaction  with  the 
telemedicine  equipment 


19 


Figure  6.  Pre-  and  post-study  score  on  satisfaction  with  current  life  situation  for  those  low  and  high  in 
satisfaction  with  the  telemedicine  equipment 


During  the  six-month  period  in  which  the  study  was  conducted,  participants  could  have  attended  up  to  25 
weekly  televideo  sessions  with  the  nurse.  As  a  way  of  gauging  the  participants’  commitment  to  the 
televideo  sessions,  we  counted  the  number  of  sessions  each  participant  participated.  We  found  a  wide 
variation  in  frequency  of  attendance,  ranging  from  none  to  21.  Reasons  offered  for  missing  sessions 
included  illness,  vacations,  and  medical  appointments.  We  hypothesized  that  those  who  attended  a  greater 
number  of  sessions  would  benefit  more  as  reflected  in  an  increase  in  knowledge  scores  or  changes  in 
attitudinal  scores  from  the  pre  to  post  test  assessment.  To  test  this  hypothesis,  we  divided  the  sample  at 
the  median  (1 1.0)  into  more  and  less  frequent  participants.  However,  we  found  no  relationship  between 
frequency  of  attendance  and  either  scores  on  the  know  ledge  test  or  the  self-report  questionnaires, 
suggesting  that  frequency  of  attendance  per  se  did  not  impact  the  effectiveness  of  the  program  on 
knowledge  or  attitudinal  measures  we  collected. 


Discussion 


It  is  important  for  the  nursing  profession  to  evaluate,  use,  and  recommend  best  practices  for  the 
application  of  technology  in  health  care  and  education.  The  results  of  this  study  showed  that  exposure  to 
and  use  of  the  telemedicine  equipment  and  the  weekly  discussion  with  the  nurse  increased  the 
participants’  feelings  of  self  efficacy.  The  increase  in  self  efficacy  from  the  outset  to  the  end  of  the  study 
period  that  was  observed  in  this  study  parallels  the  increase  in  self-efficacy  observed  in  another  study  in 
which  chronic  heart  failure  patients  who  experienced  periodic  televideo  sessions  also  increased  in  self 
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efficacy  (Grady  et  al.,  in  preparation).  Participants’  remarks  at  the  end  of  the  study  indicated  that  it  made 
them  more  attentive  to,  and  aware  of  changes  in,  their  vital  signs. 


While  there  was  no  increase  in  participants’  knowledge  about  their  disease,  the  higher  knowledge  post¬ 
test  scores  for  those  who  were  more  satisfied  with  the  telemedicine  equipment  may  suggest  that  a  more 
positive  attitude  towards  the  equipment  motivated  them  to  take  advantage  of  the  information  that  was 
available,  as  reflected  in  their  higher  test  scores. 


The  results  showed  there  were  no  overall  changes  in  participants’  perceived  quality  of  life  until 
participants’  disposition  to  the  telemedicine  technology  was  taken  into  account.  Those  who  were  more 
favorably  disposed  to  the  telemedicine  equipment  showed  an  improvement  in  their  reported  quality  of  life 
and  satisfaction  with  their  life  situation  while  the  quality  of  life  and  satisfaction  scores  for  those  less 
positive  about  the  equipment  tended  to  decrease.  It  would  seem  that  if  one  is  not  satisfied  or  accepting  of 
the  telemedicine  technology  that  tends  to  translate  into  resistance  and  the  technology  has  little  or  no  effect 
to  help.  If  one,  however,  is  accepting  and  open  to  the  technology  it  can  aid  and  abet  quality  of  life. 


The  finding  that  satisfaction  with  the  telemedicine  technology  facilitates  a  higher  quality  of  life  implies 
that  in  health  maintenance  programs  using  telemedicine  equipment,  it  is  important  that  the  equipment 
works  without  glitches.  There  were  some  problems  with  the  technology  during  this  study,  particularly  at 
the  outset,  but  we  do  not  have  sufficient  data  in  this  study  to  analyze  whether  there  was  any  relationship 
between  the  number  of  times  the  participants  experienced  problems  and  their  satisfaction  with  the 
telemedicine  equipment,  nor  whether  encountering  problems  deterred  further  participation.  Regardless  of 
any  problems  experienced,  as  indicated  in  the  results,  many  of  the  participants  were  positive  about  the 
technology. 


Conclusions 


Overall,  the  telemedicine  program  was  encouraging  in  that  participants  were  accepting  of  the  technology, 
and  participation  enhanced  their  sense  of  self-efficacy.  Another  encouraging  result  was  the  finding  that 
for  those  who  were  more  satisfied  with  the  technology,  perceived  quality  of  life  increased  from  the  start  to 
the  end  of  the  program.  These  findings  point  to  the  use  of  telemedicine  and  video  communication  as  a 
positive  alternative  to  face  to  face  health  care. 
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Task  Two)  Deliverable:  Report  outcome  analysis  of  experiments  to  evaluate  the  impact  of 

simulator-based  education: 


Goals:  Develop/conduct/evaluate  experiments  to  provide  evidence  for  best  practices  in  the  use 
of  simulation-based  education  for  Nursing  and  Allied  Health  students. 

Simulation  study  # 1  (nursing) . . . 

Simulation  study  #2  (radiography) . 
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Simulation  study  # 3 
(nursing) . 


Simulation  Study  #1 

January  23,  2007 

Nursing  Skill  Simulation  Learning  Protocol 
Background 

Simulation  has  been  used  for  decades  by  the  military,  aviation,  transportation,  and  most  nuclear  power 
industries.  In  the  United  States  as  well  as  other  countries,  organizations  have  invested  heavily  in  training 
tools,  educational  materials,  and  simulation  technology  during  the  last  decade.  United  States  Departments 
of  Defense,  Justice,  and  Health  have  expended  numerous  efforts  to  examine  the  best  ways  to  create 
training  simulations,  course  material,  and  scenarios  that  most  effectively  meet  training  needs  (1). 
Simulation  has  been  successfully  used  in  the  health  care  education  arena  as  a  teaching  strategy  for  clinical 
and  formal  education.  This  technique  has  been  used  in  nursing  schools  for  fewer  than  10  years  and  many 
nursing  schools  have  not  yet  explored  the  various  ways  in  which  human  patient  simulation  can  be  used  in 
instruction  (2). 

At  present,  the  use  of  three-dimensional  simulation  to  reproduce  life-like  experiences  in  order  to  improve 
the  training  of  health  care  professionals  is  developing  at  an  unprecedented  pace.  Several  challenges  in 
today’s  health  care  community  have  promulgated  the  need  for  cost-effective  and  efficient  innovative 
strategies  that  teach  nursing  students  about  real  world  nursing  in  a  simulated  environment.  As  health  care 
treatment  methods  continue  to  improve,  the  demand  for  qualified  health  care  professionals  has  increased 
in  the  midst  of  a  nursing  shortage.  The  capacity  to  train  new  nurses  to  meet  the  needs  of  health  care 
organizations  is  diminishing  while  greater  public  concern  for  patient  safety  exists.  Increased  patient 
acuity,  shorter  patient  hospital  stays,  and  an  increasingly  complex  clinical  environment  with  increased  use 
of  technology  are  situations  that  require  adequately  prepared  nurses.  Nursing  faculty  shortages,  greater 
numbers  of  students,  and  a  growing  knowledge  base  are  other  factors  impacting  the  preparation  of  future 
nurses.  Ill-prepared  nurses  face  fast-paced  practice  settings,  heavy  workloads,  and  limited  supervision.  It 
is  imperative,  therefore,  that  nurses  new  to  the  workforce  are  confident,  competent,  and  develop  good 
clinical  thinking  skills.  A  health  care  professional’s  ability  to  react  prudently  in  an  unexpected  situation 
is  a  critical  factor  for  positive  outcomes  to  result.  This  ability  is  learned  and  developed  over  time  with 
training,  practice,  and  repetition. 

In  nursing  education,  clinical  simulation  represents  a  revolutionary  technique  in  which  students  practice 
nursing  decision-making  and  treatment  skills  on  lifelike,  high-technology  manikins.  Simulation  is 
defined  as  a  teaching  method  in  which  learners  practice  tasks  and  processes  in  real-  world  type 
circumstances  using  models  or  virtual  reality  with  feedback  from  instructors  and  peers  (3).  It  is  believed 
that  the  role  of  practice  in  the  mastery  and  retention  of  a  skill  is  essential  while  attempting  to  learn  a  skill 
and,  after  initially  learning  it,  to  maintain  proficiency  (4). 

The  use  of  human  patient  simulators  provides  a  safe  learning  environment  that  allows  students  to  become 
better  equipped  to  handle  a  variety  of  circumstances  without  jeopardizing  real  patients.  Situations  that 
students  may  never  encounter  in  their  clinical  rotations  can  be  “simulated”  in  the  lab  where  faculty  can 
allow  students  to  make  mistakes  and  see  the  consequences  of  their  errors.  Students  can  not  only  learn 
patient-care  skills,  but  also  build  confidence  in  their  own  critical  thinking  and  decision  making  skills 
without  causing  discomfort  or  danger  to  real  patients.  Simulation  with  high-fidelity  manikins  provides  an 
opportunity  for  students  to  “think  on  their  feet”,  to  develop  reflection  and  analytical  skills,  and  to 
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consolidate  clinical  management,  and  organizational  skills.  Learners  need  sensory  input  from  the  task  and 
feedback  related  to  performance  to  reinforce  meaningful  skills  learning  (5). 

The  use  of  simulation  provides  value  to  nursing  educators  for  its  ability  to  mirror  changing  medical 
conditions  reflected  in  physical  changes  and  patient  comments.  Faculty  can  develop  complex  and 
changing  exemplar  case  study  scenarios  for  programming  the  virtual  patient’s  condition.  Such  scenarios 
help  students  develop  critical  thinking  skills  and  assessment  abilities  in  a  non-threatening  environment. 

The  use  of  scenarios  allows  students  to  assess  a  critical  health  incident  through  measurement  of  physical 
parameters  and  communication  with  a  “patient”  to  determine  appropriate  interventions,  real-time 
response,  and  evaluate  and  identify  the  need  for  further  intervention  (6).  Simulation  provides  an 
experiential  learning  strategy  for  decision-making  that  focuses  on  process,  including  the  use  of  clinical 
information  and  information-processing  sequence  rules  for  use  for  combining  information  and  decisions 
made.  The  development  of  clinical  decision-making  involving  higher  cognitive  skills  is  critical  to  the 
safety  and  outcomes  of  patients. 

Research  has  demonstrated  the  value  of  simulation  in  preparing  nurses  for  clinical  practice.  The  results 
of  research  suggest  that  reactive  simulation  training  in  nursing  education  is  more  beneficial  in  increasing 
skills  and  knowledge  than  traditional  methods  (3).  The  literature  indicates  that  compared  with  traditional 
methods,  simulation  provides  more  realistic  experiences,  enhances  the  acquisition  and  retention  of 
knowledge,  enhances  critical  thinking  and  psychomotor  skills,  increases  learner  self-confidence,  and  is 
perceived  to  be  more  satisfactory  for  the  students  (4).  Studies  have  also  shown  that  participation  in 
experiential  learning  improved  the  ease  of  transition  of  beginning  nursing  students  into  their  first  clinical 
rotation  (7). 

While  simulation  techniques  seem  to  offer  many  advantages  for  nursing  education,  several  barriers  also 
exist.  Reactive  simulation  utilization  has  a  financial  impact  on  resources  for  developing  and  maintaining 
these  programs.  Also  an  extensive  faculty  commitment  is  necessary  and  can  be  very  time-consuming.  In 
addition,  while  it  has  been  well  documented  that  simulation  is  valuable  in  preparing  nurses  for  clinical 
practice,  the  evidence  of  the  actual  effect  relating  to  preparing  nurses  for  decision-making  is  inconclusive. 
And,  perhaps  most  significantly,  the  transference  of  learning  to  actual  clinical  procedures  is  not  well- 
documented  (8). 

Technology-centric  learning  environments  with  telehealth  technology  tools  and  simulation  labs  housing 
varying  degrees  of  fidelity  may  enable  the  best  possible  learning  experience  for  nursing  students.  Schools 
of  nursing  and  other  academic  institutions  need  to  develop,  implement,  and  evaluate  simulation  used  for 
teaching  strategies  for  best  practices  in  education.  The  Division  of  Nursing  at  Mount  Aloysius  College  in 
west-central  Pennsylvania  will  conduct  a  replicated  research  study  to  further  investigate  the  effectiveness 
of  simulation  compared  to  traditional  techniques. 

2.0  Research  Design  and  Methods 

1.)  Design 

This  study  is  an  enhanced  replication  of  the  of  the  Simulation  by  Learning  Protocol  approved  and 
conducted  at  Mount  Aloysius  College  in  Spring  semester  2006.  It  is  designed  to  determine  if 
simulation-based  learning  is  more  effective  than  traditional  methods.  This  is  a  prospective, 
experimental  study  with  two  intervention  groups  using  a  post-only  control  design  The  study  will 
attempt  to  answer  the  following  research  questions: 

Is  simulation-based  training  more  effective  than  traditional  methods? 

Are  students  satisfied  with  this  method  of  learning? 

Are  faculty  satisfied  with  this  method  of  teaching? 
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Intervention 


A  training  protocol  has  been  developed  for  the  testing  of  two  skills:  foley  catheter  insertion  and 
nasogastric  tube  insertion.  Students  will  be  randomly  assigned  to  one  of  two  groups.  During  the  first  half 
of  the  semester,  students  in  Group  I  will  be  taught  nasogastric  insertion  using  reactive  simulation 
manikins  following  faculty  demonstration  on  SimMan,  an  advanced,  high-fidelity  manikin  .  Group  II 
students  will  be  instructed  in  nasogastric  insertion  using  traditional,  static  manikins.  All  students  will  be 
instructed  in  other  skills  using  the  reactive  simulation  manikin  during  this  time  prior  to  study  testing. 
Practice  sessions  will  be  scheduled  for  students  using  the  type  of  manikin  appropriate  according  to  their 
group/skill.  Open  practice  sessions  will  also  be  available  for  skills  other  than  those  designated  for  testing 
on  designated  skills.  Testing  for  NG  insertion  will  occur  at  mid-term  by  adjunct  faculty  blinded  to 
training  techniques. 

In  the  second  half  of  the  semester,  Group  1  students  will  be  taught  foley  catheter  insertion  using 
traditional,  static  manikins  while  Group  II  students  are  taught  this  procedure  using  reactive  simulation 
manikins.  Again,  all  students  will  be  instructed  in  other  skills  using  the  reactive  simulation  manikins  prior 
to  testing.  Practice  sessions  as  described  above  will  also  occur.  Testing  for  this  skill  will  occur  at  the  end 
of  the  semester. 

In  addition,  testing  outcomes  will  be  linked  to  cognitive  outcomes  resulting  from  test  questions  that  will 
be  included  on  the  final  exam.  This  analysis  will  allow  us  to  look  beyond  the  impact  of  simulator  fidelity 
on  skill  training,  and  examine  whether  simulator  fidelity  impacts  students’  understanding  and  ability  to 
reason  about  the  task. 

Testing 

All  students  will  be  tested  using  the  reactive,  simulation  manikins.  Testing  will  be  conducted  as  follows: 
four  testing  stations  will  be  set  up  with  two  faculty  persons  at  each  station.  Appointments  will  be 
scheduled  for  all  students.  Four  students  will  be  tested  simultaneously.  Approximately  one-fifth  of  the 
students  will  be  designated  as  “foils”.  These  students  will  be  tested  on  a  different  skill.  The  purpose  of  the 
foils  is  to  counter-act  any  information  passed  on  by  those  who  have  been  tested  to  those  who  have  not  that 
may  indicate  that  only  one  or  two  procedures  are  being  tested. 

The  observation  instruments  to  be  used  will  allow  a  subject  matter  expert  to  reliably  and  validly  evaluate 
the  performance  of  the  students  as  they  perform  the  skills  (Appendix  A,  B).  The  instrument  utilized  in  the 
initial  simulation  protocol  has  been  modified  somewhat  to  capture  more  discriminating  data.  Two 
questionnaires  will  be  completed  by  the  students.  One  will  measure  student  satisfaction  with  this  method 
of  learning  as  well  as  perceptions  of  self-efficacy  (Appendix  C)  and  will  be  completed  following  training. 
The  second  one  will  capture  students’  perceptions  of  the  learning  experience  (Appendix  D)  and  will  be 
completed  following  the  testing.  The  cognitively-based  questions  about  the  target  procedures  will  be 
incorporated  into  the  students’  final  exam.  Results  of  those  specific  questions  will  be  computed  as  a 
separate  score  and  used  in  this  study. 

3.0  Data  Collection  and  Statistics 
1.)  Sample  size 

Subjects  will  be  recruited  from  the  Nursing  130  seminar  class  scheduled  for  the  spring  Semester 
in  2007.  All  students  attending  this  class  will  be  invited  to  participate.  We  estimate  that  the 
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sample  of  50-60  students  will  yield  moderate  effect  sizes  (0.4-0. 6)  and  will  have  moderate  power 
(0. 6-0.7). 

2)  Statistical  Methods 

The  two  independently  completed  observer  instruments  for  each  student  will  be  compared  and 
large  differences  (more  than  2  scale  point  difference)  in  task  ratings  will  be  adjudicated.  The 
reliability  of  raters  in  terms  of  coefficient  alpha  will  be  computed  for  each  pair  of  raters  across 
common  students.  The  rating  instruments  will  be  summed  across  tasks  to  arrive  at  a  total  rating 
or  score.  The  two  total  rating  scores  for  each  student  will  be  averaged  to  produce  the  first 
performance  measure.  The  two  responses  to  the  overall  rating  item  for  each  student  will  also  be 
average  to  form  the  second  performance  variable. 

The  two  performance  measures  will  each  be  subjected  to  between-subjects  ANOVA.  Thus, 
performance  means  derived  from  the  traditional  (non-reactive  manikin)  training  condition  will  be 
contrasted  with  means  derived  from  the  experimental  (reactive  manikin)  training  condition.  A 
main  effect  indicating  that  the  experimental  condition  means  are  significantly  larger  than  the 
traditional  condition  means  will  provide  confirmation  of  the  hypothesis.  Effect  sizes  and 
confidence  intervals  will  also  be  computed  and  displayed. 

We  also  expect  that  a  similar  analysis  carried  out  for  the  self-report  attitudinal  scales  will  show 
more  positive  attitudes  for  the  experimental  training  condition.  The  students’  scores  on  the 
cognitive  questions  will  be  summed,  and  used  in  similar  between-subjects  ANOVA  as  described 
above.  A  significant  difference  in  the  mean  scores  in  favor  of  the  experimental  condition  would 
suggest  that  the  reactive  manikin  enhances  cognitive  understanding  of  the  tasks. 


Raters’  Overall  Assessment  of 
Ability  to  Perform  Nasogastric 
Procedure 

5 
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T  (df=52)  =  2  54,  p<  015 
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Students’  Assessment  of  Realism 
of  Feedback  to  Actions 


T  (df=51)  =  5.32,  p<001 


Students’  Assessment  of  Helpfulness 
of  Manniquin  Responsiveness 


T  (df=52)  =  4.59,  p<  001 
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Nursing  student  inserting  nasogastric  tube  using  a  reactive  (high  fidelity)  simulation  manikin. 


The  Impact  of  Mannequin  Simulator  Fidelity  on  the  Learning  of  Basic  Nursing  Procedures 

Introduction 


Background 

Simulation-based  experiential  learning  programs  offer  great  training  promise  across  a  variety  of 
educational  domains  and  contexts.  In  health  care,  simulation  has  been  used  for  both  formal  and  clinical 
education  (e.g.,  Gaba,  Howard,  &  Fish,  2001;  Gordon,  Wilkerson,  Shaffer,  &  Armstrong,  2001 ;  Marshall, 
Smith,  Gorman,  Krummel,  Haluck,  &  Cooley,  2001;  Yaeger,  Halamek,  Coyle,  Murphy,  Anderson,  Boye, 
Braccia,  McAuley,  &  DeSandre,  2004),  and  offers  a  realistic  hands-on  medium  for  acquiring  basic  skills. 
A  large  number  of  schools  and  training  organizations  are  presently  using  simulation-based  training  to 
complement  their  classroom  pedagogy  (Eder-Van  Hook,  2005;  Kapur  &  Steadman,  1998). 

Emerging  evidence  that  simulation-based  training  enhances  performance  leads  to  a  question 
about  the  level  of  simulator  fidelity  that  is  required  to  maximize  training  effectiveness.  High-fidelity 
simulators  are  more  life-like,  and  react  to  inputs  in  a  realistic  way,  potentially  leading  to  strong 
educational  benefits.  In  contrast,  low-fidelity  mannequins  capture  basic  anatomical  structure,  but  they  do 
not  mimic  any  of  the  physiological  and  behavioral  reactions  that  a  real  patient  would  exhibit  and  a  high- 
fidelity  mannequin  can  produce.  A  review  by  Issenberg,  McGaghie,  Petrusa,  Gordon,  &  Scalese  (2005) 
suggests  that  high-fidelity  feedback  that  accurately  mimics  realistic  human  dynamics  is  one  of  the 
primary  simulator  components  that  promotes  pedagogical  effectiveness. 
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But  utilizing  reactive  simulation  invokes  a  nontrivial  financial  impact  on  resources  for  developing 
and  maintaining  training  programs.  High-fidelity  mannequins  are  more  expensive  than  low-fidelity  ones, 
are  more  costly  to  maintain,  and  are  more  difficult  to  move  around.  Moreover,  a  time-consuming  and 
extensive  faculty  commitment  is  required  to  operate  and  maintain  high-fidelity  mannequins.  To  justify 
the  extra  financial  cost  and  other  resources  that  high-fidelity  mannequins  require,  it  is  prudent,  if  not 
necessary,  to  have  evidence  that  they  provide  better  training  than  traditional  methods  using  low-fidelity 
mannequins. 

Grady  et  al.  (in  press)  addressed  the  question  of  whether  learning  entry  level  nursing  procedures 
using  high-fidelity  reactive  mannequin  is  superior  to  learning  with  a  low-fidelity  mannequin.  The  authors 
hypothesized  that  training  supported  by  a  reactive  mannequin  simulator  that  provides  a  better  analog  to 
the  real  world  will  produce  a  better  training  milieu  and  eventuate  in  higher  performance  than  training 
supported  by  the  legacy  static  mannequin  that  does  not  provide  as  close  an  analog  to  real-world 
experiences.  They  examined  the  impact  of  high-  versus  low-fidelity  mannequin  levels  on  the  learning  of 
two  common  nursing  procedures:  nasogastric  tube  insertion  (NG)  and  indwelling  urinary  catheter 
insertion  (UC).  They  found  that  training  with  a  high-fidelity  mannequin  resulted  in  significantly  higher 
performance  than  training  with  a  low-fidelity  mannequin,  and  that  participants  expressed  more  positive 
attitudes  toward  the  high-  than  the  low-fidelity  mannequin,  especially  relative  to  its  realism  and 
responsiveness.  The  study  also  examined  the  impact  of  gender  on  the  acceptance  of  simulation-based 
training  technologies.  Results  indicated  that  males  were  more  positively  disposed  to  training  under  the 
high-fidelity  mannequin,  and  benefited  from  training  in  that  condition  more  than  did  the  female 
participants. 

Study  Objectives 

Grady  et  al.  (in  press)  reported  a  significant  effect  of  training  under  a  high  versus  low  fidelity 
mannequin,  but  the  effect  size  was  small  and  the  interaction  between  mannequin  fidelity  and  gender  was 
only  marginally  significant.  The  present  study  was  in  part  designed  to  ascertain  whether  the  Grady  et  al. 
(in  press)  results  could  be  replicated.  The  study  was  also  designed  to  explore  in  greater  depth  how 
mannequin  fidelity  impacted  each  of  the  nursing  procedures  being  trained.  Although  in  general  the 
design  and  procedure  followed  in  the  present  study  were  similar  to  that  used  in  Grady  et  al.  (in  press),  we 
changed  several  aspects  of  the  procedure  and  measurement  in  order  to  strengthen  the  study  design, 
sharpen  the  measurement  instruments,  and  develop  measures  of  sub  skills  for  each  procedure  to  learn 
whether  all  aspects  of  the  procedure  are  equally  affected  by  mannequin  fidelity  or  whether  some  aspects 
benefit  more  from  mannequin  fidelity  than  others. 

The  first  change  from  Grady  et  al.  (in  press)  was  to  develop  a  more  fine-grained  evaluation 
instrument  used  by  subject-matter  expert  observers  to  assess  performance  on  each  of  the jnursing 
procedures.  To  accomplish  this,  the  skills  involved  in  each  procedure  were  decomposed  into  discrete 
actions,  each  of  which  was  assessed  as  properly  performed  or  not  properly  performed.  Further,  for  each 
procedure,  the  actions  were  grouped  into  subcomponents  of  the  overall  procedure.  In  contrast,  the 
evaluation  instrument  used  in  Grady  et  al.  (in  press)  included  a  smaller  number  of  more  global  items,  each 
assessing  an  aspect  of  the  procedure.  Performance  on  each  item  was  evaluated  using  a  Likert-type  scale. 
By  decomposing  the  skills  down  into  finer-grained  steps,  we  felt  we  would  be  able  to  more  precisely 
pinpoint  the  impact  of  any  differences  that  occurred  in  training  under  a  low  versus  a  high  fidelity 
mannequin. 

A  second  change  involved  practice.  A  number  of  participants  in  the  Grady  et  al.  (in  press)  study 
mentioned  on  the  post  evaluation  questionnaire  that  they  would  have  liked  more  opportunity  to  practice 
the  skill  on  the  mannequin  than  was  available  in  the  class  period  in  which  it  was  trained.  In  the  present 
study,  participants  were  afforded  the  opportunity  to  practice  the  skill  (using  the  same  level  of  mannequin 


29 


on  which  they  were  trained)  at  a  designated  time.  The  opportunity  to  practice  the  skill  strengthened  any 
differences  that  might  occur  in  training  under  the  two  different  types  of  mannequins. 

Finally,  in  the  Grady  et  al  (in  press)  study,  the  evaluation  for  both  skills  was  done  during  the  final 
exam  period.  In  this  study,  the  evaluations  for  the  two  skills  were  conducted  at  separate  times,  closer  to 
when  they  were  trained. 

In  addition,  we  also  wanted  to  examine  whether  mannequin  fidelity  level  impacted  participants’ 
knowledge  about  the  procedures  as  reflected  in  their  scores  on  relevant  skill-oriented  questions  items  in 
the  final  exam.  (For  example,  in  the  NG  procedure,  what  is  the  appropriate  action  to  take  if  the  patient 
gasps,  coughs,  or  does  not  respond  vocally  to  questions  when  the  nurse  is  inserting  the  tube?)  Although 
as  a  written  test,  the  final  exam  necessarily  assessed  knowledge  (what  to  do)  rather  than  skill  (how  to  do 
it),  we  hypothesized  that  the  more  lifelike  aspects  of  the  high-fidelity  mannequin  may  have  motivated 
better  retention  of  knowledge.  This  would  be  consistent  with  the  Alinier  and  colleagues’  (2006)  findings 
that  students  who  received  supplementary  scenario-based  training  using  an  intermediate  fidelity  had 
higher  scores  on  the  Objective  Structured  Clinical  Examination  than  students  who  followed  the  normal 
curriculum. 


Method 


Participants 

The  participants  in  this  study  were  students  from  first  year  nursing  education  classes  who  signed 
the  consent  form.  A  sample  of  54  participants,  consisting  of  43  women  and  1 1  men,  was  obtained. 

Independent  Variables  and  Design 

The  independent  variables  and  design  were  adapted  from  Grady  et  al.  (in  press).  Two  nursing 
procedures  were  studied:  nasogastric  tube  insertion  (NG)  and  indwelling  urinary  catheter  insertion  (UC). 

The  first  independent  variable  was  training  mannequin  fidelity  manipulated  over  the  levels  of  low 
and  high  fidelity.  A  non-reactive  partial  torso  mannequin  focusing  on  the  anatomical  area  of  interest  was 
employed  for  the  low  fidelity  level,  whereas  a  reactive  full  body  mannequin  was  used  for  the  high  fidelity 
level.  For  the  NG  procedure  a  head  and  chest  model  that  allows  a  tube  to  be  inserted  in  either  nostril  and 
for  the  UC  procedure  a  lower  torso  catheterization  model  that  allows  for  a  tube  insertion  were  the  partial 
torso  models  used  in  the  low  fidelity  condition.  Neither  of  these  partial  torso  models  was  reactive  in  any 
way.  The  Laerdal™  SimMan™  Universal  Patient  Simulator  was  used  for  both  NG  and  UC  training  in  the 
high  fidelity  condition.  The  SimMan™  mannequin,  in  addition  to  being  anatomically  correct,  is  reactive 
to  various  student  actions  such  as  saying  “ouch”  and  gagging.  If  the  procedure  is  carried  out  correctly,  for 
the  UC  procedure  “urine”  is  obtained  and  for  the  NG  procedure  “body  fluids”  are  obtained. 

As  in  the  Grady  et  al.  (in  press)  study,  the  second  independent  variable  was  the  participant’s 

gender. 


The  students  were  trained  on  both  procedures  as  part  of  the  course  curriculum.  Two  experimental 
conditions  were  created.  A  random  half  of  the  participants  were  assigned  to  an  experimental  condition  in 
which  NG  training  was  performed  with  a  high-fidelity  mannequin  followed  by  UC  training  with  a  low- 
fidelity  mannequin.  The  remaining  participants  experienced  a  condition  in  which  NG  training  was 
performed  with  a  low-fidelity  mannequin  followed  by  UC  training  with  a  high-fidelity  mannequin. 
Pedagogical  constraints  precluded  the  counter-balancing  of  the  order  in  which  participants  were  trained 
on  the  two  nursing  procedures. 
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Dependent  Variables 

A  targeted  observation  instrument  was  derived  from  materials  traditionally  used  to  evaluate  NG 
and  UC  performance.  Each  instrument  consisted  of  a  number  of  actions  necessary  to  perform  the  specific 
nursing  procedure.  Each  action  was  accompanied  by  a  bivariate  yes-no  scale.  If  the  action  was  observed 
to  be  correctly  performed,  yes  was  checked  to  indicate  successful  completion  of  that  action.  If  the  action 
was  not  performed  or  was  performed  incorrectly,  no  was  checked.  The  NG  observation  instrument  was 
comprised  of  35  items,  and  the  UC  instrument  contained  39  items.  The  items  were  developed  so  that 
subscales  assessing  specific  aspects  of  the  two  nursing  procedures  could  be  measured.  Some  of  the 
subscale  categories  addressing  pre  and  post  insertion  activities  were  the  same  for  both  procedures,  but 
necessarily  the  subcategories  that  reflected  the  procedure  itself  were  different.  For  the  NG  procedure  the 
items  were  grouped  to  assess  five  subskills:  prior-to-procedure  preparation,  bedside  preparation,  tube 
insertion,  verify  and  secure,  and  documentation.  For  the  UC  procedure  the  items  were  grouped  to  assess 
six  subskills:  prior-to-procedure  preparation,  bedside  preparation,  sterile  field  preparation,  catheter 
insertion,  post-catheter  care,  and  documentation.  For  both  the  overall  scores  and  the  subskill  scores,  a 
participant’s  score  was  the  percentage  of  items  that  were  checked  yes. 

Two  experienced  observers  rated  each  participant  on  each  procedure.  The  observers  were  blind 
to  the  condition  under  which  the  participant  trained.  To  assess  the  inter-rater  reliability  for  the  assessment 
of  each  nursing  procedure,  the  mean  scores  derived  from  rater  Vs  observations  and  the  mean  scores 
derived  from  rater  2’s  observations  across  all  participants  were  correlated  and  analyzed  to  yield 
Coefficient  Alpha  reliability  statistics  (Cronbach,  1951).  In  the  same  manner  the  inter-rater  reliability 
assessments  for  the  subskills  associated  with  each  nursing  procedure  were  derived.  The  interrater 
reliabilities  for  the  overall  and  the  subskill  scores  are  shown  in  Table  1.  The  reliability  for  the  overall 
score  was  high  for  both  procedures.  With  the  exception  of  UC  documentation,  all  the  subskills  were 
assessed  with  adequate  to  high  reliability.  Although  the  inter-rater  reliability  of  documentation  is  quite 
low  it  does  not  appear  to  detrimentally  affect  overall  reliability  of  the  UC  instrument.  Moreover,  none  of 
the  analyses  examines  the  subskills  individually  so  this  one  instance  of  low  reliability  can  be  tolerated. 


Table  1 .  Interrater  Reliabilities  for  the  Two  Nursing  Procedures 


NG 

Reliability 

UC 

Reliability 

Overall 

0.86 

Overall 

0.88 

Subskills 

Sub  skills 

Prior  to  procedure 
prep 

0.89 

Prior  to  procedure 

PreP 

0.88 

Bedside  preparation 

0.74 

Bedside  preparation 

0.63 

Sterile  field 
preparation 

0.54 

Tube  insertion 

0.47 

Catheter  insertion 

0.80 

Verify  and  secure 

0.87 

Post  catheter  care 

0.80 

Documentation 

0.84 

Documentation 

0.17 

For  each  procedure  two  self-report  instruments  described  in  Grady  et  al.  (in  press)  were 
administered.  The  Post-Training  Questionnaire,  which  assessed  attitudes  about  the  mannequin-based  skill 
training,  was  completed  by  all  participants  at  the  end  of  the  NG  training  and  again  after  the  UC  training. 
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The  instrument  was  comprised  of  eight  items  and  had  a  reported  reliability  coefficient  of  0.88.  The  Post- 
Evaluation  Questionnaire,  which  addressed  participants’  assessment  of  their  performance  on  the  test,  their 
confidence  in  their  ability  to  perform  the  procedure,  and  their  opinions  about  the  training  they  received, 
was  completed  after  the  test  on  the  NG  procedure  and  again  after  the  test  for  the  UC  procedure.  Because 
the  Post-Evaluation  Questionnaire  assessed  multiple  factors,  positive  intercorrelations  among  the  items 
were  not  expected;  thus,  Coefficient  Alpha  reliability  was  not  computed. 

Procedure 

As  part  of  their  introductory  nursing  course  participants  were  trained  on  the  NG  procedure  and 
several  weeks  later  the  UC  procedure.  The  training  was  done  as  part  of  the  regular  classroom/lab 
curriculum  and  all  participants  were  afforded  some  hands-on  experience.  At  the  completion  of  each 
training  period  participants  completed  the  Post-Training  Questionnaire. 

Following  the  training,  but  prior  to  testing,  practice  sessions  were  schedule  in  which  the 
participants  had  the  opportunity  to  practice  the  nursing  procedure  using  the  type  of  mannequin 
appropriate  to  their  group  and  nursing  procedure.  For  each  procedure,  participants  were  tested  within 
four  weeks  after  training.  (Testing  on  the  NG  procedure  was  done  as  part  of  the  midterm  examination. 
Testing  on  the  UC  procedure  was  done  as  part  of  the  final  exam.)  All  testing  was  completed  using  high 
fidelity  mannequins.  The  assessment  for  each  participant  lasted  approximately  30  minutes. 

At  testing  time  a  participant  entered  a  testing  station  and  was  instructed  to  perform  the  targeted 
nursing  procedures.  Two  subject  matter  expert  (SME)  observers  were  present  at  the  station  and  each 
independently  evaluated  the  participant’s  performance  using  the  observational  assessment  instrument.  At 
the  conclusion  of  testing  the  participant  completed  the  Post-Evaluation  Questionnaire.  At  the  conclusion 
of  the  course  participants  were  debriefed  as  to  the  nature  of  the  experiment. 


Results 


Performance  Results 

Overall  Measures:  One  purpose  of  this  study  was  to  see  if  we  could  replicate  the  Grady  et  al.  (in 
press)  findings  of  a  main  effect  due  to  fidelity  and  a  gender  by  fidelity  interaction.  A  comparable  analysis 
for  the  present  study  is  summarized  in  Fig.  1.  The  gender  (2)  X  fidelity  (2)  mixed  ANOVA  failed  to 
reveal  any  significant  main  effects  or  interaction,  Fs  <  1.0,  ns.  Initially,  then,  the  results  of  this  study  do 
not  appear  to  replicate  the  Grady  et  al.  (in  press)  findings.  However,  inspection  of  the  means  pattern  in 
Fig.  1  indicates  that  the  fidelity  effect  is  present  for  the  women  but  not  for  the  men  in  the  sample.  A  one¬ 
way  ANOVA  for  mannequin  fidelity  for  only  the  women  in  the  sample  showed  that  women  who  trained 
with  the  high  fidelity  mannequin  (M  —  82.49)  outperformed  those  who  trained  with  the  low  fidelity 
mannequin  (M  =  78.14,  F(l,  40)  =  4.28,  p  <  .05).  No  such  effect  was  present  for  the  men.  Thus,  in  this 
study  the  finding  reported  by  Grady  et  al.  (in  press)  that  training  with  a  high  fidelity  reactive  mannequin 
resulted  in  higher  performance  scores  was  replicated  for  the  female  component  of  the  sample.  This  result 
is  in  contrast  to  the  Grady  et  al  (in  press)  study,  where  the  advantage  of  the  high-fidelity  mannequin  was 
stronger  for  the  males  than  the  females.  The  implications  of  these  results  will  be  taken  up  in  the  next 
section  in  more  detail.  Moreover,  because  a  significant  gender  effect  was  not  found,  it  was  dropped  as  an 
independent  variable  in  all  subsequent  contrasts. 
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Figure  1.  Performance  Scores  the  Low  and  High  Fidelity  Mannequin  Training  Conditions  for  Male  and 

Female  Participants 

A  second  purpose  of  the  present  study  was  to  look  more  closely  at  how  training  mannequin 
fidelity  impacted  performance  on  each  of  the  two  procedures.  For  this,  1-way  between-subjects 
ANOVAs  were  computed  for  the  NG  and  UC  procedures.  As  Fig.  2  shows,  for  the  NG  procedure 
participants  who  trained  with  the  high  fidelity  mannequin  earned  higher  test  scores  than  those  who  trained 
with  the  low  fidelity  mannequin,  F(1 , 52)  =  7.44,  p  <  .01 .  The  same  analysis  for  the  UC  procedure,  also 
depicted  in  Fig.  2,  failed  to  yield  significant  results  (F  <  1 .0,  n.s).  Thus,  there  is  evidence  that  the  fidelity 
of  the  mannequin  did  not  impact  the  participants’  training  on  the  two  procedures  in  the  same  way. 


Figure  2.  Performance  Scores  for  the  Low  and  High  Fidelity  Mannequin  Training  Conditions  for  the  NG 

and  UC  Procedures 

Subscale  measures.  To  continue  our  investigation  on  how  the  fidelity  of  the  training  mannequin 
impacts  the  nursing  procedures  being  trained,  we  examined  the  subskill  scores  for  the  NG  and  UC 
evaluation  instruments.  Scores  from  the  five  subskills  comprising  the  NG  assessment  instrument  were 
subjected  to  a  MANOVA  with  fidelity  as  the  independent  variable.  A  significant  multivariate  effect  was 
revealed  for  fidelity,  F(5,  48)  =  3.09,  p  <  .02,  and  from  Fig.  3  we  can  see  that  for  every  subskill 
performance  was  higher  when  training  was  performed  with  the  high  than  low  fidelity  mannequin.  The 
between-subject  tests  for  the  individual  subskill  scores  showed  that  the  primary  source  of  the  multivariate 
effect  was  derived  from  tube  insertion,  F(l,  54)  =  8.71,  p  <  .01,  and  verification  of  placement  and 
securing  of  tube,  F(l,  54)  =  13.71,  p  <  .01.  In  other  words,  it  was  these  two  subskills  of  the  NG  procedure 
that  were  most  facilitated  by  training  with  the  high  fidelity  mannequin. 


33 


■  Low  Fidelity  ■  High  Fidelity 


100 
8  ^ 
|  60 
1  40 


m  ei  n  i  ;i 


PrlorPrep  Bedside  Tube  Insert  Verify 
Prep  Secure 

Subskills 


Document 


Figure  3.  Subskill  Scores  for  the  Low  and  High  Fidelity  Mannequin  Training  Conditions  for  the  NG 

Procedure 

When  a  similar  MANOVA  was  computed  for  the  scores  from  the  six  subskills  comprising  the  UC 
evaluation  instrument,  no  multivariate  effect  for  fidelity  was  observed  and  no  individual  subscales  scores 
proved  significant  either.  The  means  for  this  analysis  can  be  found  in  Fig.  4.  The  result  of  this  analysis 
mirrors  the  nonsignificant  fidelity  effect  for  the  overall  UC  procedure  score. 
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Figure  4.  Subskill  Scores  for  the  Low  and  High  Fidelity  Mannequin  Training  Conditions  for  the  UC 

Procedure 


Self-report  Instrument  Results 

Post  Training  Instrument.  Paralleling  the  evaluation  of  the  performance  data,  we  subjected  the 
self-report  data  collected  following  the  NG  and  UC  training  to  a  within-subjects  ANOVA  averaging  over 
nursing  procedures  to  contrast  low  versus  high  fidelity  mannequin  levels  for  each  of  the  eight  post¬ 
training  self-report  items.  Only  one  contrast  for  the  item  “confidence  to  perform  this  procedure  on  a  real 
patient  without  supervision'’  was  significant,  F(l,  51)  =  9.08,  p  <  .01.  As  expected  participants  reported 
higher  confidence  when  using  the  high  fidelity  (M  =  3.48)  than  low  fidelity  (M  =  2.98)  mannequin. 

To  look  separately  at  the  two  procedures,  the  post-training  self-report  data  for  each  of  the  two 
procedures  were  subjected  to  a  MANOVA  with  mannequin  fidelity  as  the  independent  variable. 
Consistent  with  the  performance  results,  the  MANOVA  for  NG  training  returned  a  significant 
multivariate  effect  for  fidelity,  F(8,  43)  =  5.62,  p  <  .01,  indicating  that  participants  who  trained  with  the 
high  fidelity  mannequin  held  more  positive  attitudes  toward  the  NG  training  than  participants  who  trained 
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with  the  low  fidelity  mannequin.  Inspection  of  the  individual  items,  depicted  in  Fig  5,  showed  that  four 
items  accounted  for  the  significant  multivariate  effect.  Participants  felt  that:  the  high  fidelity  mannequin 
provided  more  realistic  feedback  to  their  actions,  F(l,  50)  =  26.39,  p  <  .01;  the  responsiveness  of  the  high 
fidelity  mannequin  helped  them  learn  how  to  do  the  procedure,  F(l,  50)  =  24.79,  p  <  .01;  the  high  fidelity 
mannequin  increased  their  motivation  to  learn,  F(l,  50)  =  8.47,  p  <  .01;  and  working  with  the  high  fidelity 
mannequin  improved  the  whole  course  experience,  t(50)  =  1.54,  p  <  .04,  one-tail. 
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Attitude  To  Training  Items 


Figure  5.  The  Four  Items  of  the  Attitudes  Toward  Training  Instrument  Providing  the  Principal  Support 
for  the  Overall  Mannequin  Fidelity  Multivariate  Effect  for  the  NG  Procedure 

Compatible  with  the  lack  of  performance  results  reported  above  for  UC,  the  MANOVA  on  the 
post-training  items  for  UC  training  did  not  yield  a  significant  multivariate  effect,  F  <  1 .0,  ns.  There  is  no 
difference  in  attitudes  when  training  with  a  high  versus  a  low  fidelity  mannequin  for  UC  skills. 

Post  Evaluation  Instrument.  This  instrument  assessed  the  participants’  perceptions  of  their  test 
performance,  their  confidence  in  their  ability  to  perform  the  procedure,  and  their  opinions  about  the 
training  they  received.  As  indicated  previously,  the  items  comprising  the  Post-Evaluation  instrument 
were  not  intended  to  assess  a  single  dimension,  so  each  of  the  10  items  was  analyzed  separately.  As  with 
the  post-training  items  a  within-subjects  analysis  ANOVA  averaging  over  procedures  was  conducted  to 
contrast  low  versus  high  fidelity  mannequin  levels  for  each  of  the  10  post-evaluation  items.  These 
within-subjects  analyses  did  not  reveal  any  reliable  differences  between  low  and  high  mannequin  fidelity. 

Again  using  mannequin  fidelity  as  the  independent  variable,  a  one-way  ANOVA  was  conducted 
for  each  item  for  the  NG  procedure  and  for  the  UC  procedure.  None  of  the  ANOVAs  yielded  significant 
results. 


Each  of  the  two  post-evaluation  questionnaires  included  an  open-ended  question  asking  what 
was  the  most  difficult  part  of  the  procedure.  For  the  NG  procedure,  the  responses  were  varied.  A 
number  of  participants  mentioned  issues  related  to  placement  of  the  tube;  others  mentioned  the  difficulty 
of  remembering  all  the  steps  in  order.  The  participants’  responses  did  not  differ  by  level  of  mannequin 
fidelity  used  in  training. 

For  the  UC  procedure,  over  60  percent  of  the  student  mentioned  difficulties  associated  with 
maintaining  a  sterile  field.  This  consensus  about  the  difficulty  aspect  was  most  striking  in  the  low  fidelity 
training  condition  in  which  74  percent  of  the  26  participants  offered  this  response.  (In  the  high  fidelity 
mannequin  group,  55  percent  of  the  22  partic  ipants  gave  this  response).  This  aspect  of  the  UC 
procedure  was  partially  captured  in  one  of  the  subscales  of  the  performance  instrument:  sterile  field 
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preparation.  It  is  interesting  to  note,  as  shown  in  Fig  4,  that  despite  their  concern,  performance  on  this 
aspect  of  the  procedure  was  quite  high,  and  there  was  no  difference  between  the  low-  and  high-fidelity 
groups. 


Final  Examination.  As  part  of  this  study,  we  planned  to  examine  the  impact  of  mannequin 
fidelity  on  relevant  components  of  final  examination.  However,  the  students’  scores  on  the  relevant  items 
were  uniformly  high  (mean  scores  >  90%.)  creating  a  ceiling  effect,  with  insufficient  variance  to  conduct 
a  meaningful  analysis. 


Discussion 


Performance  Results 

One  of  the  goals  of  this  study  was  to  replicate  the  Grady  et  al.  (in  press)  findings  that  high  fidelity 
mannequins  offer  superior  training  outcomes  when  compared  to  low  fidelity  mannequins.  The  results 
replicated  the  Grady  et  al.  (in  press)  findings,  but  only  for  the  female  component  of  the  sample.  In 
contrast,  the  Grady  et  al.  (in  press)  findings  were  based  on  the  whole  sample,  and  the  advantage  of  the 
high  fidelity  mannequins  was  even  stronger  for  males  than  females.  The  differing  results  for  the  males  in 
the  two  samples  may  have  more  to  do  with  the  small  number  of  males  present  in  the  samples  than  any  real 
interaction  with  mannequin  fidelity.  There  were  only  about  a  dozen  male  participants  in  each  sample. 

The  separate  analyses  of  the  individual  procedures  suggests  the  benefits  of  the  high-fidelity 
mannequin  may  not  accrue  equally  across  procedures.  The  present  results  show  that  the  training  of  the 
NG  procedure  was  positively  and  significantly  impacted  by  a  high  fidelity  mannequin,  but  this  was  not  so 
for  training  of  the  UC  procedure.  Moreover,  it  was  the  specific  subskills  of  tube  insertion  and  verification 
of  placement  and  securing  of  tube  within  the  NG  procedure  that  benefited  most  from  training  with  the 
high  fidelity  mannequin.  We  would  like  to  argue  that  these  two  subskills  are  dynamic  in  nature  and  thus 
are  supported  by  the  reactivity/responsiveness  of  the  high  fidelity  mannequin,  but  then  we  are  hard 
pressed  to  explain  the  lack  of  effect  with  the  corresponding  UC  subskills,  catheter  insertion  and  post¬ 
catheter  care.  At  this  point  we  can  conclude  that  all  basic  nursing  procedures,  and  more  specifically  the 
requisite  subskills  comprising  a  procedure,  will  not  benefit  equally  from  training  with  high  fidelity 
mannequins.  More  research  on  particular  procedures  is  needed  to  better  understand  the  benefits  of  a  high- 
fidelity  mannequin. 

Self  Report  Results 

The  performance  results  indicating  that  the  advantages  of  training  with  a  high  fidelity  mannequin 
accrue  for  the  NG  procedure  but  not  the  UC  procedure  are  reflected  in  the  participants’  perceptions  of  the 
effectiveness  of  training.  We  can  speculate  that  in  the  NG  procedure  the  ‘ouch’  and  gurgling  that  the  high 
fidelity  mannequin  provides  is  useful  feedback  to  the  students  about  tube  placement,  which  is  one  of  the 
concerns  the  participants  expressed  in  the  open-ended  question  on  the  post-evaluation  questionnaire.  The 
results  for  the  UC  procedure  are  somewhat  surprising  in  that  the  high-fidelity  mannequin  does  provide 
“urine”  when  the  tube  is  correctly  placed.  However,  the  students  were  clearly  more  focused  on  the 
difficulty  of  maintaining  a  sterile  field,  and  the  high  fidelity  mannequin  provides  no  specific  advantage  in 
that  respect  (except  maybe  for  motivation). 

Limitations 

One  limitation  imposed  by  doing  this  study  in  the  context  of  ongoing  classroom  education  is  that 
we  were  not  able  to  counter-balance  the  order  of  the  NG  and  UC  training.  The  students  do  the  NG 
training  first,  so  it  is  possible  that  there  is  an  order  effect  here  that  produced  the  significant  NG  results 
and/or  lack  of  results  for  the  UC  procedure. 
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The  small  male  sample  size  imposes  a  difficulty  in  the  interpretation  of  gender  results,  and 
specifically  why  the  males  showed  an  advantage  under  the  high-fidelity  mannequin  in  the  Grady  et  al.  (in 
press)  study,  but  not  the  present  one.  Additional  research  with  a  larger  sample  of  male  participants  is 
needed  to  clarify  this  question.. 

Finally,  we  note  that  this  study  only  addressed  performance  closely  following  training.  Studies 
have  shown  that  participation  in  experiential  learning  improves  the  ease  of  transition  of  beginning  nursing 
students  into  their  first  clinical  rotation  (Ham  &  O’Rourke,  2004).  Peteani  (2004)  cites  evidence  that 
students  exhibit  increased  autonomy  and  self-confidence  when  delivering  patient  care  after  practicing 
with  a  high-fidelity  simulator.  Participants  in  the  present  study  reported  feeling  more  confident  in  their 
ability  to  perform  the  procedure  on  a  real  patient  without  supervision  when  they  trained  under  a  high 
fidelity  mannequin.  But  is  there  any  evidence  that  those  who  train  with  a  high  fidelity  mannequin  are 
more  proficient  at  doing  the  insertion  on  a  real  patient?  Addressing  this  question  requires  follow-up  when 
the  students  begin  their  first  post-training  clinical  rotation  in  which  they  are  performing  these  procedures. 
Because  other  factors  may  intervene,  it  would  be  most  useful  if  these  students  could  be  followed  as  soon 
as  possible  after  the  training. 


Conclusions 

The  question  of  when,  and  under  what  conditions,  a  high-fidelity  mannequin  is  advantageous 
over  a  low-fidelity  mannequin  has  implications  in  nursing  departments  for  decisions  about  whether  to 
allocate  resources  for  the  purchase  of  high  fidelity  mannequins  and,  if  they  are  available,  in  what  aspects 
of  the  nursing  curriculum  they  are  most  advantageous.  The  partial  replication  of  the  Grady  et  al.  (in 
press)  study  lends  further  credibility  to  positive  training  effects  of  high  fidelity  mannequins  and  justifies 
to  some  extent  the  extra  cost  of  such  training  mannequins.  The  finding  that  the  advantage  accrued  to  one 
basic  skill  (NG)  but  not  another  (UC)  suggests  the  need  to  clarify  for  which  basic  skills  the  use  of  a  high- 
fidelity  mannequin  is  most  advantageous.  It  may  also  be  that  the  advantages  of  training  with  a  high- 
fidelity  mannequin  are  more  significant  for  training  complex  skills  rather  than  basic  ones.  We  are 
addressing  this  question  in  our  current  research. 
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The  Impact  of  Mannequin  Simulator  Fidelity  on  the  Learning  of  Basic 

Nursing  Procedures 

Analysis  Report 

Introduction 


Background 

Simulation-based  experiential  learning  programs  offer  great  training  promise  across  a  variety  of 
educational  domains  and  contexts.  In  health  care,  simulation  has  been  used  for  both  formal  and  clinical 
education  (e.g.,  Gaba,  Howard,  &  Fish,  2001;  Gordon,  Wilkerson,  Shaffer,  &  Armstrong,  2001;  Marshall, 
Smith,  Gorman,  Krummel,  Haluck,  &  Cooley,  2001;  Yaeger,  Halamek,  Coyle,  Murphy,  Anderson,  Boye, 
Braccia,  McAuley,  &  DeSandre,  2004),  and  offers  a  realistic  hands-on  medium  for  acquiring  basic  skills. 
A  large  number  of  schools  and  training  organizations  are  presently  using  simulation-based  training  to 
complement  their  classroom  pedagogy  (Eder-Van  Hook,  2005;  Kapur  &  Steadman,  1998). 

Emerging  evidence  that  simulation-based  training  enhances  performance  leads  to  a  question 
about  the  level  of  simulator  fidelity  that  is  required  to  maximize  training  effectiveness.  High-fidelity 
simulators  are  more  life-like,  and  react  to  inputs  in  a  realistic  way,  potentially  leading  to  strong 
educational  benefits.  In  contrast,  low-fidelity  mannequins  capture  basic  anatomical  structure,  but  they  do 
not  mimic  any  of  the  physiological  and  behavioral  reactions  that  a  real  patient  would  exhibit  and  a  high- 
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fidelity  mannequin  can  produce.  A  review  by  Issenberg,  McGaghie,  Petrusa,  Gordon,  &  Scalese  (2005) 
suggests  that  high-fidelity  feedback  that  accurately  mimics  realistic  human  dynamics  is  one  of  the 
primary  simulator  components  that  promotes  pedagogical  effectiveness. 

But  utilizing  reactive  simulation  invokes  a  nontrivial  financial  impact  on  resources  for  developing 
and  maintaining  training  programs.  High-fidelity  mannequins  are  more  expensive  than  low-fidelity  ones, 
are  more  costly  to  maintain,  and  are  more  difficult  to  move  around.  Moreover,  a  time-consuming  and 
extensive  faculty  commitment  is  required  to  operate  and  maintain  high-fidelity  mannequins.  To  justify 
the  extra  financial  cost  and  other  resources  that  high-fidelity  mannequins  require,  it  is  prudent,  if  not 
necessaiy,  to  have  evidence  that  they  provide  better  training  than  traditional  methods  using  low-fidelity 
mannequins. 

Grady  et  al.  (in  press)  addressed  the  question  of  whether  learning  entry  level  nursing  procedures 
using  high-fidelity  reactive  mannequin  is  superior  to  learning  with  a  low-fidelity  mannequin.  The  authors 
hypothesized  that  training  supported  by  a  reactive  mannequin  simulator  that  provides  a  better  analog  to 
the  real  world  will  produce  a  better  training  milieu  and  eventuate  in  higher  performance  than  training 
supported  by  the  legacy  static  mannequin  that  does  not  provide  as  close  an  analog  to  real-world 
experiences.  They  examined  the  impact  of  high-  versus  low- fidelity  mannequin  levels  on  the  learning  of 
two  common  nursing  procedures:  nasogastric  tube  insertion  (NG)  and  indwelling  urinary  catheter 
insertion  (UC).  They  found  that  training  with  a  high-fidelity  mannequin  resulted  in  significantly  higher 
performance  than  training  with  a  low-fidelity  mannequin,  and  that  participants  expressed  more  positive 
attitudes  toward  the  high-  than  the  low-fidelity  mannequin,  especially  relative  to  its  realism  and 
responsiveness.  The  study  also  examined  the  impact  of  gender  on  the  acceptance  of  simulation-based 
training  technologies.  Results  indicated  that  males  were  more  positively  disposed  to  training  under  the 
high-fidelity  mannequin,  and  benefited  from  training  in  that  condition  more  than  did  the  female 
participants. 

Study  Objectives 

Grady  et  al.  (in  press)  reported  a  significant  effect  of  training  under  a  high  versus  low  fidelity 
mannequin,  but  the  effect  size  was  small  and  the  interaction  between  mannequin  fidelity  and  gender  was 
only  marginally  significant.  The  present  study  was  in  part  designed  to  ascertain  whether  the  Grady  et  al. 
(in  press)  results  could  be  replicated.  The  study  was  also  designed  to  explore  in  greater  depth  how 
mannequin  fidelity  impacted  each  of  the  nursing  procedures  being  trained.  Although  in  general  the 
design  and  procedure  followed  in  the  present  study  were  similar  to  that  used  in  Grady  et  al.  (in  press),  we 
changed  several  aspects  of  the  procedure  and  measurement  in  order  to  strengthen  the  study  design, 
sharpen  the  measurement  instruments,  and  develop  measures  of  subskills  for  each  procedure  to  learn 
whether  all  aspects  of  the  procedure  are  equally  affected  by  mannequin  fidelity  or  whether  some  aspects 
benefit  more  from  mannequin  fidelity'  than  others. 

The  first  change  from  Grady  et  al.  (in  press)  was  to  develop  a  more  fine-grained  evaluation 
instrument  used  by  subject-matter  expert  observers  to  assess  performance  on  each  of  the  nursing 
procedures.  To  accomplish  this,  the  skills  involved  in  each  procedure  were  decomposed  into  discrete 
actions,  each  of  which  was  assessed  as  properly  performed  or  not  properly  performed.  Further,  for  each 
procedure,  the  actions  were  grouped  into  subcomponents  of  the  overall  procedure.  In  contrast,  the 
evaluation  instrument  used  in  Grady  et  al.  (in  press)  included  a  smaller  number  of  more  global  items,  each 
assessing  an  aspect  of  the  procedure.  Performance  on  each  item  was  evaluated  using  a  Likert-type  scale. 
By  decomposing  the  skills  down  into  finer-grained  steps,  we  felt  we  would  be  able  to  more  precisely 
pinpoint  the  impact  of  any  differences  that  occurred  in  training  under  a  low  versus  a  high  fidelity 
mannequin. 
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A  second  change  involved  practice.  A  number  of  participants  in  the  Grady  et  al.  (in  press)  study 
mentioned  on  the  post  evaluation  questionnaire  that  they  would  have  liked  more  opportunity  to  practice 
the  skill  on  the  mannequin  than  was  available  in  the  class  period  in  which  it  was  trained.  In  the  present 
study,  participants  were  afforded  the  opportunity  to  practice  the  skill  (using  the  same  level  of  mannequin 
on  which  they  were  trained)  at  a  designated  time.  The  opportunity  to  practice  the  skill  strengthened  any 
differences  that  might  occur  in  training  under  the  two  different  types  of  mannequins. 

Finally,  in  the  Grady  et  al  (in  press)  study,  the  evaluation  for  both  skills  was  done  during  the  final 
exam  period.  In  this  study,  the  evaluations  for  the  two  skills  were  conducted  at  separate  times,  closer  to 
when  they  were  trained. 

In  addition,  we  also  wanted  to  examine  whether  mannequin  fidelity  level  impacted  participants’ 
knowledge  about  the  procedures  as  reflected  in  their  scores  on  relevant  skill-oriented  questions  items  in 
the  final  exam.  (For  example,  in  the  NG  procedure,  what  is  the  appropriate  action  to  take  if  the  patient 
gasps,  coughs,  or  does  not  respond  vocally  to  questions  when  the  nurse  is  inserting  the  tube?)  Although 
as  a  written  test,  the  final  exam  necessarily  assessed  knowledge  (what  to  do)  rather  than  skill  (how  to  do 
it),  we  hypothesized  that  the  more  lifelike  aspects  of  the  high-fidelity  mannequin  may  have  motivated 
better  retention  of  knowledge.  This  would  be  consistent  with  the  Alinier  and  colleagues’  (2006)  findings 
that  students  who  received  supplementary  scenario-based  training  using  an  intermediate  fidelity  had 
higher  scores  on  the  Objective  Structured  Clinical  Examination  than  students  who  followed  the  normal 
curriculum. 


Method 


Participants 

The  participants  in  this  study  were  students  from  first  year  nursing  education  classes  who  signed 
the  consent  form.  A  sample  of  54  participants,  consisting  of  43  women  and  1 1  men,  was  obtained. 

Independent  Variables  and  Design 

The  independent  variables  and  design  were  adapted  from  Grady  et  al.  (in  press).  Two  nursing 
procedures  were  studied:  nasogastric  tube  insertion  (NG)  and  indwelling  urinary  catheter  insertion  (UC). 

The  first  independent  variable  was  training  mannequin  fidelity  manipulated  over  the  levels  of  low 
and  high  fidelity.  A  non-reactive  partial  torso  mannequin  focusing  on  the  anatomical  area  of  interest  was 
employed  for  the  low  fidelity  level,  whereas  a  reactive  full  body  mannequin  was  used  for  the  high  fidelity 
level.  For  the  NG  procedure  a  head  and  chest  model  that  allows  a  tube  to  be  inserted  in  either  nostril  and 
for  the  UC  procedure  a  lower  torso  catheterization  model  that  allows  for  a  tube  insertion  were  the  partial 
torso  models  used  in  the  low  fidelity  condition.  Neither  of  these  partial  torso  models  was  reactive  in  any 
way.  The  Laerdal™  SimMan™  Universal  Patient  Simulator  was  used  for  both  NG  and  UC  training  in  the 
high  fidelity  condition.  The  SimMan™  mannequin,  in  addition  to  being  anatomically  correct,  is  reactive 
to  various  student  actions  such  as  saying  “ouch”  and  gagging.  If  the  procedure  is  carried  out  correctly,  for 
the  UC  procedure  “urine”  is  obtained  and  for  the  NG  procedure  “body  fluids”  are  obtained. 

As  in  the  Grady  et  al.  (in  press)  study,  the  second  independent  variable  was  the  participant’s 

gender. 


The  students  were  trained  on  both  procedures  as  part  of  the  course  curriculum.  Tw  o  experimental 
conditions  were  created.  A  random  half  of  the  participants  were  assigned  to  an  experimental  condition  in 
which  NG  training  was  performed  with  a  high-fidelity  mannequin  followed  by  UC  training  with  a  low- 
fidelity  mannequin.  The  remaining  participants  experienced  a  condition  in  which  NG  training  was 
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performed  with  a  low-fidelity  mannequin  followed  by  UC  training  with  a  high-fidelity  mannequin. 
Pedagogical  constraints  precluded  the  counter-balancing  of  the  order  in  which  participants  were  trained 
on  the  two  nursing  procedures. 

Dependent  Variables 

A  targeted  observation  instrument  was  derived  from  materials  traditionally  used  to  evaluate  NG 
and  UC  performance.  Each  instrument  consisted  of  a  number  of  actions  necessary  to  perform  the  specific 
nursing  procedure.  Each  action  was  accompanied  by  a  bivariate  yes-no  scale.  If  the  action  was  observed 
to  be  correctly  performed,  yes  was  checked  to  indicate  successful  completion  of  that  action.  If  the  action 
was  not  performed  or  was  performed  incorrectly,  no  was  checked.  The  NG  observation  instrument  was 
comprised  of  35  items,  and  the  UC  instrument  contained  39  items.  The  items  were  developed  so  that 
subscales  assessing  specific  aspects  of  the  two  nursing  procedures  could  be  measured.  Some  of  the 
subscale  categories  addressing  pre  and  post  insertion  activities  were  the  same  for  both  procedures,  but 
necessarily  the  subcategories  that  reflected  the  procedure  itself  were  different.  For  the  NG  procedure  the 
items  were  grouped  to  assess  five  subskills:  prior-to-procedure  preparation,  bedside  preparation,  tube 
insertion,  verily  and  secure,  and  documentation.  For  the  UC  procedure  the  items  were  grouped  to  assess 
six  subskills:  prior-to-procedure  preparation,  bedside  preparation,  sterile  field  preparation,  catheter 
insertion,  post-catheter  care,  and  documentation.  For  both  the  overall  scores  and  the  subskill  scores,  a 
participant’s  score  was  the  percentage  of  items  that  were  checked  yes. 

Two  experienced  observers  rated  each  participant  on  each  procedure.  The  observers  were  blind 
to  the  condition  under  which  the  participant  trained.  To  assess  the  inter-rater  reliability  for  the  assessment 
of  each  nursing  procedure,  the  mean  scores  derived  from  rater  Vs  observations  and  the  mean  scores 
derived  from  rater  2’s  observations  across  all  participants  were  correlated  and  analyzed  to  yield 
Coefficient  Alpha  reliability  statistics  (Cronbach,  1951).  In  the  same  manner  the  inter-rater  reliability 
assessments  for  the  subskills  associated  with  each  nursing  procedure  were  derived.  The  interrater 
reliabilities  for  the  overall  and  the  subskill  scores  are  shown  in  Table  1 .  The  reliability  for  the  overall 
score  was  high  for  both  procedures.  With  the  exception  of  UC  documentation,  all  the  subskills  were 
assessed  with  adequate  to  high  reliability.  Although  the  inter-rater  reliability  of  documentation  is  quite 
low  it  does  not  appear  to  detrimentally  affect  overall  reliability  of  the  UC  instrument.  Moreover,  none  of 
the  analyses  examines  the  subskills  individually  so  this  one  instance  of  low  reliability  can  be  tolerated. 

Table  1 .  Interrater  Reliabilities  for  the  Two  Nursing  Procedures 


NG 

Reliability 

UC 

Reliability 

Overall 

0.86 

Overall 

0.88 

Subskills 

Subskills 

Prior  to  procedure  prep 

0.89 

Prior  to  procedure  prep 

0.88 

Bedside  preparation 

0.74 

Bedside  preparation 

0.63 

Sterile  field  preparation 

0.54 

Tube  insertion 

0.47 

Catheter  insertion 

0.80 

Verify  and  secure 

0.87 

Post  catheter  care 

0.80 

Documentation 

0.84 

Documentation 

0.17 

For  each  procedure  two  self-report  instruments  described  in  Grady  et  al.  (in  press)  were 
administered.  The  Post-Training  Questionnaire,  which  assessed  attitudes  about  the  mannequin-based  skill 
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training,  was  completed  by  all  participants  at  the  end  of  the  NG  training  and  again  after  the  UC  training. 
The  instrument  was  comprised  of  eight  items  and  had  a  reported  reliability  coefficient  of  0.88.  The  Post- 
Evaluation  Questionnaire,  which  addressed  participants’  assessment  of  their  performance  on  the  test,  their 
confidence  in  their  ability  to  perform  the  procedure,  and  their  opinions  about  the  training  they  received, 
was  completed  after  the  test  on  the  NG  procedure  and  again  after  the  test  for  the  UC  procedure.  Because 
the  Post-Evaluation  Questionnaire  assessed  multiple  factors,  positive  intercorrelations  among  the  items 
were  not  expected;  thus.  Coefficient  Alpha  reliability  was  not  computed. 

Procedure 

As  part  of  their  introductory  nursing  course  participants  were  trained  on  the  NG  procedure  and 
several  weeks  later  the  UC  procedure.  The  training  was  done  as  part  of  the  regular  classroom/lab 
curriculum  and  all  participants  were  afforded  some  hands-on  experience.  At  the  completion  of  each 
training  period  participants  completed  the  Post-Training  Questionnaire. 

Following  the  training,  but  prior  to  testing,  practice  sessions  were  schedule  in  which  the 
participants  had  the  opportunity  to  practice  the  nursing  procedure  using  the  type  of  mannequin 
appropriate  to  their  group  and  nursing  procedure.  For  each  procedure,  participants  were  tested  within 
four  weeks  after  training.  (Testing  on  the  NG  procedure  was  done  as  part  of  the  midterm  examination. 
Testing  on  the  UC  procedure  was  done  as  part  of  the  final  exam.)  All  testing  was  completed  using  high 
fidelity  mannequins.  The  assessment  for  each  participant  lasted  approximately  30  minutes. 

At  testing  time  a  participant  entered  a  testing  station  and  was  instructed  to  perform  the  targeted 
nursing  procedures.  Two  subject  matter  expert  (SME)  observers  were  present  at  the  station  and  each 
independently  evaluated  the  participant’s  performance  using  the  observational  assessment  instrument.  At 
the  conclusion  of  testing  the  participant  completed  the  Post-Evaluation  Questionnaire.  At  the  conclusion 
of  the  course  participants  w  ere  debriefed  as  to  the  nature  of  the  experiment. 

Results 


Performance  Results 

mannequin  resulted  in  higher  performance  scores  was  replicated  for  the  female  component  of 
the  Overall  Measures:  One  purpose  of  this  study  was  to  see  if  we  could  replicate  the  Grady  et  al.  (in 
press)  findings  of  a  main  effect  due  to  fidelity  and  a  gender  by  fidelity  interaction.  A  comparable  analysis 
for  the  present  study  is  summarized  in  Fig.  1.  The  gender  (2)  X  fidelity  (2)  mixed  ANOVA  failed  to 
reveal  any  significant  main  effects  or  interaction,  Fs  <  1 .0,  ns.  Initially,  then,  the  results  of  this  study  do 
not  appear  to  replicate  the  Grady  et  al.  (in  press)  findings.  However,  inspection  of  the  means  pattern  in 
Fig.  1  indicates  that  the  fidelity  effect  is  present  for  the  women  but  not  for  the  men  in  the  sample.  A  one¬ 
way  ANOVA  for  mannequin  fidelity  for  only  the  women  in  the  sample  showed  that  women  who  trained 
with  the  high  fidelity  mannequin  (M  =  82.49)  outperformed  those  who  trained  with  the  low  fidelity 
mannequin  (M  =  78.14,  F(l,  40)  =  4.28,  p  <  .05).  No  such  effect  w;as  present  for  the  men.  Thus,  in  this 
study  the  finding  reported  by  Grady  et  al.  (in  press)  that  training  with  a  high  fidelity  reactive  sample. 

This  result  is  in  contrast  to  the  Grady  et  al  (in  press)  study,  where  the  advantage  of  the  high-fidelity 
mannequin  was  stronger  for  the  males  than  the  females.  The  implications  of  these  results  will  be  taken  up 
in  the  next  section  in  more  detail.  Moreover,  because  a  significant  gender  effect  was  not  found  it  was 
dropped  as  an  independent  variable  in  all  subsequent  contrasts. 
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Figure  1.  Performance  Scores  the  Low  and  High  Fidelity  Mannequin  Training  Conditions  for  Male  and 

Female  Participants 

A  second  purpose  of  the  present  study  was  to  look  more  closely  at  how  training  mannequin 
fidelity  impacted  performance  on  each  of  the  two  procedures.  For  this,  1-way  between-subjects 
ANOVAs  were  computed  for  the  NG  and  UC  procedures.  As  Fig.  2  shows,  for  the  NG  procedure 
participants  who  trained  with  the  high  fidelity  mannequin  earned  higher  test  scores  than  those  who  trained 
with  the  low  fidelity  mannequin,  F(l,  52)  =  7.44,  p  <  .01.  The  same  analysis  for  the  UC  procedure,  also 
depicted  in  Fig.  2,  failed  to  yield  significant  results  (F  <  1.0,  n.s).  Thus,  there  is  evidence  that  the  fidelity 
of  the  mannequin  did  not  impact  the  participants’  training  on  the  two  procedures  in  the  same  way. 


Figure  2.  Performance  Scores  for  the  Low  and  High  Fidelity  Mannequin  Training  Conditions  for  the  NG 

and  UC  Procedures 

Subscale  measures.  To  continue  our  investigation  on  how  the  fidelity  of  the  training  mannequin 
impacts  the  nursing  procedures  being  trained,  we  examined  the  subskill  scores  for  the  NG  and  UC 
evaluation  instruments.  Scores  from  the  five  subskills  comprising  the  NG  assessment  instrument  were 
subjected  to  a  MANOVA  with  fidelity  as  the  independent  variable.  A  significant  multivariate  effect  was 
revealed  for  fidelity,  F(5,  48)  =  3.09,  p  <  .02,  and  from  Fig.  3  we  can  see  that  for  every  subskill 
performance  was  higher  when  training  was  performed  with  the  high  than  low  fidelity  mannequin.  The 
between-subject  tests  for  the  individual  subskill  scores  showed  that  the  primary  source  of  the  multivariate 
effect  was  derived  from  tube  insertion,  F(l,  54)  =  8.71,  p  <  .01,  and  verification  of  placement  and 
securing  of  tube,  F(l,  54)  =  13.71,  p  <  .01.  In  other  words,  it  was  these  two  subskills  of  the  NG  procedure 
that  were  most  facilitated  by  training  with  the  high  fidelity  mannequin. 
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Figure  3.  Subskill  Scores  for  the  Low  and  High  Fidelity  Mannequin  Training  Conditions  for  the  NG 

Procedure 

When  a  similar  MANOVA  was  computed  for  the  scores  from  the  six  subskills  comprising  the  UC 
evaluation  instrument,  no  multivariate  effect  for  fidelity  was  observed  and  no  individual  subscales  scores 
proved  significant  either.  The  means  for  this  analysis  can  be  found  in  Fig.  4.  The  result  of  this  analysis 
mirrors  the  nonsignificant  fidelity  effect  for  the  overall  UC  procedure  score. 
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Figure  4.  Subskill  Scores  for  the  Low  and  High  Fidelity  Mannequin  Training  Conditions  for  the  UC 

Procedure 


Self-report  Instrument  Results 

Post  Trainine  Instrument.  Paralleling  the  evaluation  of  the  perfonnance  data,  we  subjected  the 
self-report  data  collected  following  the  NG  and  UC  training  to  a  within-subjects  ANOVA  averaging  over 
nursing  procedures  to  contrast  low  versus  high  fidelity  mannequin  levels  for  each  of  the  eight  post¬ 
training  self-report  items.  Only  one  contrast  for  the  item  “confidence  to  perform  this  procedure  on  a  real 
patient  without  supervision”  was  significant,  F(  1,51)  =  9.08,  p  <  .01.  As  expected  participants  reported 
higher  confidence  when  using  the  high  fidelity  (M  =  3.48)  than  low  fidelity  (M  =  2.98)  mannequin. 

To  look  separately  at  the  two  procedures,  the  post-training  self-report  data  for  each  of  the  two 
procedures  were  subjected  to  a  MANOVA  with  mannequin  fidelity  as  the  independent  variable. 
Consistent  with  the  performance  results,  the  MANOVA  for  NG  training  returned  a  significant 
multivariate  effect  for  fidelity,  F(8,  43)  =  5.62,  p  <  .01 ,  indicating  that  participants  who  trained  with  the 
high  fidelity  mannequin  held  more  positive  attitudes  toward  the  NG  training  than  participants  who  trained 
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with  the  low  fidelity  mannequin.  Inspection  of  the  individual  items,  depicted  in  Fig  5,  showed  that  four 
items  accounted  for  the  significant  multivariate  effect.  Participants  felt  that:  the  high  fidelity  mannequin 
provided  more  realistic  feedback  to  their  actions,  F(l,  50)  =  26.39,  p  <  .01;  the  responsiveness  of  the  high 
fidelity  mannequin  helped  them  learn  how  to  do  the  procedure,  F(l,  50)  =  24.79,  p  <  .01;  the  high  fidelity 
mannequin  increased  their  motivation  to  learn,  F(l,  50)  =  8.47,  p  <  .01;  and  working  with  the  high  fidelity 
mannequin  improved  the  whole  course  experience,  t(50)  =  1.54,  p  <  .04,  one-tail. 


■  Low  ■  High 


Attitude  To  Training  Items 


Figure  5.  The  Four  Items  of  the  Attitudes  Toward  Training  Instrument  Providing  the  Principal  Support 
for  the  Overall  Mannequin  Fidelity  Multivariate  Effect  for  the  NG  Procedure 

Compatible  with  the  lack  of  performance  results  reported  above  for  UC,  the  MANOVA  on  the 
post-training  items  for  UC  training  did  not  yield  a  significant  multivariate  effect,  F  <  1.0,  ns.  There  is  no 
difference  in  attitudes  when  training  with  a  high  versus  a  low  fidelity  mannequin  for  UC  skills. 

Post  Evaluation  Instrument.  This  instrument  assessed  the  participants’  perceptions  of  their  test 
performance,  their  confidence  in  their  ability  to  perfonn  the  procedure,  and  their  opinions  about  the 
training  they  received.  As  indicated  previously,  the  items  comprising  the  Post-Evaluation  instrument 
were  not  intended  to  assess  a  single  dimension,  so  each  of  the  10  items  was  analyzed  separately.  As  with 
the  post-training  items  a  within-subjects  analysis  ANOVA  averaging  over  procedures  was  conducted  to 
contrast  low  versus  high  fidelity  mannequin  levels  for  each  of  the  10  post-evaluation  items.  These 
within-subjects  analyses  did  not  reveal  any  reliable  differences  between  low  and  high  mannequin  fidelity. 

Again  using  mannequin  fidelity  as  the  independent  variable,  a  one-way  ANOVA  was  conducted 
for  each  item  for  the  NG  procedure  and  for  the  UC  procedure.  None  of  the  ANOVAs  yielded  significant 
results. 


Each  of  the  two  post-evaluation  questionnaires  included  an  open-ended  question  asking  what 
was  the  most  difficult  part  of  the  procedure.  For  the  NG  procedure,  the  responses  were  varied.  A 
number  of  participants  mentioned  issues  related  to  placement  of  the  tube;  others  mentioned  the  difficulty 
of  remembering  all  the  steps  in  order.  The  participants’  responses  did  not  differ  by  level  of  mannequin 
fidelity  used  in  training. 

For  the  UC  procedure,  over  60  percent  of  the  student  mentioned  difficulties  associated  with 
maintaining  a  sterile  field.  This  consensus  about  the  difficulty  aspect  was  most  striking  in  the  low  fidelity 
training  condition  in  which  74  percent  of  the  26  participants  offered  this  response.  (In  the  high  fidelity 
mannequin  group,  55  percent  of  the  22  participants  gave  this  response).  This  aspect  of  the  UC 
procedure  was  partially  captured  in  one  of  the  subscales  of  the  performance  instrument:  sterile  field 
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preparation.  It  is  interesting  to  note,  as  shown  in  Fig  4,  that  despite  their  concern,  performance  on  this 
aspect  of  the  procedure  was  quite  high,  and  there  was  no  difference  between  the  low-  and  high-fidelity 
groups. 


Final  Examination.  As  part  of  this  study,  we  planned  to  examine  the  impact  of  mannequin 
fidelity  on  relevant  components  of  final  examination.  However,  the  students’  scores  on  the  relevant  items 
were  uniformly  high  (mean  scores  >  90%.)  creating  a  ceiling  effect,  with  insufficient  variance  to  conduct 
a  meaningful  analysis. 


Discussion 


Performance  Results 

One  of  the  goals  of  this  study  was  to  replicate  the  Grady  et  al.  (in  press)  findings  that  high  fidelity 
mannequins  offer  superior  training  outcomes  when  compared  to  low  fidelity  mannequins.  The  results 
replicated  the  Grady  et  al.  (in  press)  findings,  but  only  for  the  female  component  of  the  sample.  In 
contrast,  the  Grady  et  al.  (in  press)  findings  were  based  on  the  wliole  sample,  and  the  advantage  of  the 
high  fidelity  mannequins  was  even  stronger  for  males  than  females.  The  differing  results  for  the  males  in 
the  tw  o  samples  may  have  more  to  do  with  the  small  number  of  males  present  in  the  samples  than  any  real 
interaction  with  mannequin  fidelity.  There  were  only  about  a  dozen  male  participants  in  each  sample. 

The  separate  analyses  of  the  individual  procedures  suggests  the  benefits  of  the  high-fidelity 
mannequin  may  not  accrue  equally  across  procedures.  The  present  results  show  that  the  training  of  the 
NG  procedure  was  positively  and  significantly  impacted  by  a  high  fidelity  mannequin,  but  this  was  not  so 
for  training  of  the  UC  procedure.  Moreover,  it  was  the  specific  subskills  of  tube  insertion  and  verification 
of  placement  and  securing  of  tube  within  the  NG  procedure  that  benefited  most  from  training  with  the 
high  fidelity  mannequin.  We  would  like  to  argue  that  these  two  subskills  are  dynamic  in  nature  and  thus 
are  supported  by  the  reactivity/responsiveness  of  the  high  fidelity  mannequin,  but  then  we  are  hard 
pressed  to  explain  the  lack  of  effect  with  the  corresponding  UC  subskills,  catheter  insertion  and  post¬ 
catheter  care.  At  this  point  we  can  conclude  that  all  basic  nursing  procedures,  and  more  specifically  the 
requisite  subskills  comprising  a  procedure,  will  not  benefit  equally  from  training  with  high  fidelity 
mannequins.  More  research  on  particular  procedures  is  needed  to  better  understand  the  benefits  of  a  high- 
fidelity  mannequin. 

Self  Report  Results 

The  performance  results  indicating  that  the  advantages  of  training  with  a  high  fidelity  mannequin 
accrue  for  the  NG  procedure  but  not  the  UC  procedure  are  reflected  in  the  participants’  perceptions  of  the 
effectiveness  of  training.  We  can  speculate  that  in  the  NG  procedure  the  ‘ouch1  and  gurgling  that  the  high 
fidelity  mannequin  provides  is  useful  feedback  to  the  students  about  tube  placement,  which  is  one  of  the 
concerns  the  participants  expressed  in  the  open-ended  question  on  the  post-evaluation  questionnaire.  The 
results  for  the  UC  procedure  are  somewhat  surprising  in  that  the  high-fidelity  mannequin  does  provide 
“urine”  when  the  tube  is  correctly  placed.  However,  the  students  were  clearly  more  focused  on  the 
difficulty  of  maintaining  a  sterile  field,  and  the  high  fidelity  mannequin  provides  no  specific  advantage  in 
that  respect  (except  maybe  for  motivation). 

Limitations 

One  limitation  imposed  by  doing  this  study  in  the  context  of  ongoing  classroom  education  is  that 
wre  were  not  able  to  counter-balance  the  order  of  the  NG  and  UC  training.  The  students  do  the  NG 
training  first,  so  it  is  possible  that  there  is  an  order  effect  here  that  produced  the  significant  NG  results 
and/or  lack  of  results  for  the  UC  procedure. 
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The  small  male  sample  size  imposes  a  difficulty  in  the  interpretation  of  gender  results,  and 
specifically  why  the  males  showed  an  advantage  under  the  high-fidelity  mannequin  in  the  Grady  et  al.  (in 
press)  study,  but  not  the  present  one.  Additional  research  with  a  larger  sample  of  male  participants  is 
needed  to  clarify  this  question.. 

Finally,  we  note  that  this  study  only  addressed  performance  closely  following  training.  Studies 
have  shown  that  participation  in  experiential  learning  improves  the  ease  of  transition  of  beginning  nursing 
students  into  their  first  clinical  rotation  (Ham  &  O’Rourke,  2004).  Peteani  (2004)  cites  evidence  that 
students  exhibit  increased  autonomy  and  self-confidence  when  delivering  patient  care  after  practicing 
with  a  high-fidelity  simulator.  Participants  in  the  present  study  reported  feeling  more  confident  in  their 
ability  to  perform  the  procedure  on  a  real  patient  without  supervision  when  they  trained  under  a  high 
fidelity  mannequin.  But  is  there  any  evidence  that  those  who  train  with  a  high  fidelity  mannequin  are 
more  proficient  at  doing  the  insertion  on  a  real  patient?  Addressing  this  question  requires  follow-up  when 
the  students  begin  their  first  post-training  clinical  rotation  in  which  they  are  performing  these  procedures. 
Because  other  factors  may  intervene,  it  would  be  most  useful  if  these  students  could  be  followed  as  soon 
as  possible  after  the  training. 


Conclusions 

The  question  of  when,  and  under  what  conditions,  a  high-fidelity  mannequin  is  advantageous 
over  a  low-fidelity  mannequin  has  implications  in  nursing  departments  for  decisions  about  whether  to 
allocate  resources  for  the  purchase  of  high  fidelity  mannequins  and,  if  they  are  available,  in  what  aspects 
of  the  nursing  curriculum  they  are  most  advantageous.  The  partial  replication  of  the  Grady  et  al.  (in 
press)  study  lends  further  credibility  to  positive  training  effects  of  high  fidelity  mannequins  and  justifies 
to  some  extent  the  extra  cost  of  such  training  mannequins.  The  finding  that  the  advantage  accrued  to  one 
basic  skill  (NG)  but  not  another  (UC)  suggests  the  need  to  clarity  for  which  basic  skills  the  use  of  a  high- 
fidelity  mannequin  is  most  advantageous.  It  may  also  be  that  the  advantages  of  training  with  a  high- 
fidelity  mannequin  are  more  significant  for  training  complex  skills  rather  than  basic  ones.  We  are 
addressing  this  question  in  our  current  research. 
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Simulation  Study  #  2 


Radiography  Simulation  Learning  Protocol 


Background 

Simulation  has  been  used  for  decades  by  the  military,  aviation,  transportation,  and  most  nuclear  power 
industries.  In  the  United  States  as  well  as  other  countries,  organizations  have  invested  heavily  in  training 
tools,  educational  materials,  and  simulation  technology  in  the  last  decade.  US  Departments  of  Defense, 
Justice,  and  Health  have  expended  numerous  efforts  to  examine  the  best  ways  to  create  training 
simulations,  course  material,  and  scenarios  that  most  effectively  meet  training  needs  (1).  Simulation  has 
been  successfully  used  in  the  health  care  education  arena  as  a  teaching  strategy  for  clinical  and  formal 
education. 

Simulator-based  education  has  the  potential  to  positively  impact  our  ability  to  train  students  in  allied 
health  professional  fields,  such  as  Diagnostic  Medical  Imaging  and  Radiography.  Allied  health 
professionals  trained  in  these  fields  are  technologists  who  work  directly  with  patients  and  physicians  as 
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they  perform  diagnostic  testing  procedures.  Because  of  their  diagnostic  utility  and  relative  safety  as  a 
diagnostic  procedure,  these  fields  are  expected  to  grow  significantly  over  the  next  decade.  To  fill  the 
predicted  need  for  qualified  allied  health  care  providers,  efficient  and  cost-effective  methods  are  needed 
to  train  these  professionals. 

According  to  the  Bureau  of  Labor  Statistics,  employment  of  radiology  technologists  is  predicted  to  grow 
faster  than  average  between  2006  to  2016  (2).  The  general  population  is  aging  and  the  demand  for 
diagnostic  imaging  is  increasing.  As  the  population  changes  so  does  the  profession.  The  field  of  medical 
imaging  is  evolving  technologically  due  to  the  advancement  of  computers  which  has  prompted  traditional 
imaging  to  become  digital.  Images  are  archived  and  sent  to  networked  facilities  via  a  Picture  Archiving 
and  Communication  System  (PACS).  PACS  and  digital  imaging  has  recently  been  implemented  into  the 
national  registry  exam  for  radiologic  technologists  but  little  information  has  been  found  pertaining  to 
medical  imaging  students’  education  in  PACS  for  digital  imaging.  The  American  Society  of  Radiologic 
Technologists  (ASRT)  Radiology  Curriculum  suggests  that  digital  imaging  acquisition  and  display  be  a 
key  content  area  and  for  programs  to  provide  an  understanding  of  the  components  and  principles  of  digital 
imaging  systems  (3).  The  curriculum  also  recommends  students  understand  the  basics  of  digital  imaging 
systems,  digital  image  display,  architecture  of  PACS,  teleradiography,  post-processing  capabilities,  and 
display  monitors. 

Demands  on  health  care  providers  and  medical  educators  to  improve  patient  safety  and  care  have 
increased.  The  concept  of ‘learning  by  doing”  has  become  less  acceptable.  Technology  can  provide 
alternative  methods  to  gain  procedural  experience.  Web-based  education,  virtual  reality  and  high-fidelity 
human  patient  simulation  are  examples  of  state-of-the-art  approaches  to  educating  students  in  health 
related  fields  (4). 

Largely  delivered  using  the  traditional  classroom  format,  digital  imaging  education  for  allied  health 
students  has  depended  on  the  availability  of  appropriate  equipment  for  practice  in  the  actual  clinical 
settings,  which  may  or  may  not  be  available  to  students  in  all  geographic  locations.  Technological 
advances  in  equipment  have  provided  additional  tools  for  educators,  but  the  literature  is  lacking  in 
evidence-based,  best  practice  approaches  to  training  techniques  using  advanced  educational  technologies 
in  the  classroom  setting. 

Although  there  is  paucity  in  the  literature  addressing  best  practices  in  the  use  of  technology,  the  Health 
Care  Industry  Advisory  Council  and  the  American  Society  of  Radiologic  Technologists  have  joined 
forces  to  establish  best  practices  and  recommendations  for  improving  new  skills  acquisition  for  radiologic 
technologists.  One  area  that  was  addressed  is  to  enhance  the  entry-level  radiologic  technologist 
educational  process  to  prepare  individuals  for  the  influence  of  technology  and  the  importance  of  life  long 
learning.  A  technologist’s  initial  education  is  key  to  later  skills  acquisition  and  commitment  to  life  long 
learning.  The  primary  education  also  forms  the  core  skills  needed  to  grasp  the  concepts  on  which  new 
technologies  are  based.  The  International  Society  of  Radiographers  and  Radiological  Technologists  views 
entry-level  education  as  a  starting  point  in  its  educational  program  guidelines  (5). 

Several  challenges  in  today’s  health  care  community  have  promulgated  the  need  for  cost-effective  and 
efficient  innovative  strategies  that  teach  radiography  students  about  real  world  radiography  in  a  simulated 
environment.  The  capacity  to  train  new  staff  to  meet  the  needs  of  health  care  organizations  is  diminishing 
while  greater  public  concern  for  patient  safety  exists.  Increased  patient  acuity,  shorter  patient  hospital 
stays,  and  an  increasingly  complex  clinical  environment  with  increased  use  of  technology  are  situations 
that  require  adequately  prepared  radiographers.  Radiography  faculty  shortages  and  a  growing  increase  in 
the  required  knowledge  base  are  other  factors  impacting  future  radiography  preparation.  Ill-prepared 
radiographers  face  fast-paced  practice  settings,  heavy  workloads,  and  limited  supervision.  It  is 
imperative,  therefore,  that  radiographers  new  to  the  workforce  are  confident,  competent,  and  develop 
good  clinical  thinking  skills.  A  health  care  professional’s  ability  to  react  prudently  in  an  unexpected 
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situation  is  a  critical  factor  for  positive  outcomes  to  result.  This  ability  is  learned  and  developed  over 
time  with  training,  practice,  and  repetition. 

Radiography  education  is  very  similar  to  that  of  nursing  education  in  which  clinical  simulation  represents 
a  revolutionary  technique  in  which  students  practice  patient  positioning  on  manikins.  Simulation  is 
defined  as  a  teaching  method  in  which  learners  practice  tasks  and  processes  in  real-  world  type 
circumstances  using  models  or  virtual  reality  with  feedback  from  instructors  and  peers  (6).  It  is  believed 
that  the  role  of  practice  in  the  mastery  of  a  skill  is  essential  while  attempting  to  learn  a  skill  and,  after 
initially  learning  it,  in  the  retention  to  maintain  proficiency  (7).  While  simulation  might  be  applicable  to 
image  interpretation  fields  like  radiology,  personnel  can  often  train  using  archived  images  of  real  patients 
(8). 

Currently,  there  are  limited  simulation  devices  available  in  radiography.  Simulation  software  focusing  on 
PACS,  computed  radiography,  and  virtual  positioning  are  obtainable.  Computer  simulation  attempts  to 
reproduce  real-life  situations  and  asks  the  student  to  provide  data  that  may  alter  the  outcome  of  the 
procedure.  In  the  health  professions,  allowing  a  student  to  attempt  patient  care  without  “experimenting-’ 
on  actual  patients  is  ideal.  Simulations  can  bridge  the  gap  from  abstract  knowledge  presented  in  class  to 
actual  performance  by  letting  the  student  learn  the  difference  between  good  and  poor  decisions  in  patient 
care.  Simulations  also  can  bring  about  higher-order  synthesis  and  analysis  skills  (9). 

The  use  of  computerized  simulation  software  and  manikins  provides  a  safe  learning  environment  that 
allows  students  to  become  better  equipped  to  handle  a  variety  of  circumstances  without  jeopardizing  real 
patients.  Situations  that  students  may  never  encounter  in  their  clinical  rotations  can  be  “simulated'’  in  the 
lab  where  faculty  can  allow  students  to  make  mistakes  and  see  the  consequences  of  their  errors.  Students 
can  not  only  learn  positioning  skills  and  practice  utilizing  the  PACS  and  workstation,  but  also  build 
confidence  in  their  own  critical  thinking  and  decision  making  skills  without  causing  discomfort  or  danger 
to  real  patients.  The  use  of  equipment  and  software  can  be  a  viable  educational  tool  to  teach  multiple 
skills  related  to  anatomy/pathology,  image  quality,  patient  positioning,  etc.  Computer  skills  are 
considered  mandatory  for  today’s  radiologic  technologists,  and  digital  imaging  skills  have  been  identified 
as  among  the  most  important  skills  for  these  allied  health  professionals,  regardless  of  geographic  location 
(10).  As  with  the  education  of  all  health  care  providers,  patient  safety,  the  reduction  of  medical  errors, 
and  improving  department  efficiency  by  embracing  current  and  new  technologies  will  continue  to  drive 
the  development  of  educational  strategies. 

Technology-centric  learning  environments  with  simulation  software  and  PACS  may  enable  the  best 
possible  learning  experience  for  radiography  students.  Schools  of  radiography  and  other  academic 
institutions  need  to  develop,  implement,  and  evaluate  simulation  used  for  teaching  strategies  for  best 
practices  in  education.  The  Medical  Imaging  and  Radiation  Sciences  Department  at  Mount  Aloysius 
College  in  west-central  Pennsylvania  will  conduct  a  research  study  to  investigate  the  effectiveness  of 
simulation  compared  to  traditional  techniques. 

Research  Design  and  Methods 
Design 

The  study  is  designed  to  determine  if  simulation-based  learning  is  more  effective  than  traditional 
methods.  This  is  a  prospective,  experimental  study  with  two  intervention  groups  using  a  post-only 
control  For  this  design  order  is  nested  within  the  original  factor,  training  ty  pe,  to  produce  four 
experimental  conditions.  At  one  nested  level  of  training  type,  students  experience  training  module  one 
using  the  radiology  simulation  environment  and,  when  that  training  is  complete,  experience  training 
module  two  using  the  legacy  training  procedure.  In  contrast,  at  nested  level  two,  students  experience 
training  module  one  using  the  legacy  training  procedure  and,  when  that  training  is  complete,  experience 
training  module  two  using  the  radiology  simulator.  Thus  each  student  in  the  study  has  experience  with 
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the  radiology  simulation  environment  and  the  legacy  training  procedure.  For  the  analysis,  collapsing  over 
training  modules  we  can  compare  each  student’s  performance  with  the  radiology  simulation  environment 
to  his  or  her  performance  with  the  legacy  procedure  as  a  within-subjects  analysis.  Within-subjects 
analyses  are  more  powerful  (i.e.,  able  to  detect  smaller  differences)  as  each  student  serves  as  his  or  her 
own  control. 

The  study  will  attempt  to  answer  the  following  research  questions: 

Is  simulation-based  training  more  effective  than  traditional  method? 

i.  For  positioning? 

ii.  For  analysis  and  interpretation? 

Are  students  satisfied  with  this  method  of  learning? 

Are  faculty  members  satisfied  with  this  method  of  teaching? 


We  hypothesize  that  the  simulation/PACS-based  training  will  be  more  effective  than  the  traditional 
(legacy)  training  for  both  patient  positioning  and  film  interpretation 

Intervention 

A  training  protocol  will  be  developed  for  the  testing  of  two  skills:  procedure  and  interpretation.  Students 
will  be  assigned  to  one  of  two  groups  depending  upon  the  laboratory  section  in  which  they  enroll.  During 
the  first  half  of  the  semester,  students  in  Group  I  will  be  taught  procedures  and  film  interpretation  using 
simulation  software  and  the  PACS.  Group  II  students  will  be  instructed  in  procedures  and  interpretation 
using  the  legacy  training  method.  Testing  for  procedures  and  film  interpretation  will  occur  following  the 
training  at  or  prior  to  the  midpoint  of  the  class. 

In  the  second  half  of  the  semester,  Group  1  students  will  be  taught  procedures  and  film  interpretation 
using  the  legacy  training  method  while  Group  2  students  will  be  taught  procedures  and  film  interpretation 
using  the  simulation  software  and  PACS.  Testing  for  procedures  and  film  interpretation  will  occur  during 
the  last  half  of  the  semester  prior  to  the  final  exam. 

Testing 

Testing  will  occur  during  the  semester.  Students  in  both  groups  will  receive  the  same  exams.  There  will 
be  two  testing  components:  a  written  exam  and  laboratory  simulation.  The  written  exams  on  the  content 
discussed  in  class  will  be  completed  by  students  in  both  groups.  (Appendix  A).  The  written  exam  will 
have  questions  pertaining  to  positioning  (procedure),  anatomy,  and  film  critique  (interpretation).  During 
the  week  of  the  written  exam,  each  student  in  the  class  will  make  an  appointment  with  instructor  and/or  a 
subject  matter  expert  (SME)  to  simulate  a  procedure  or  part  of  a  procedure  utilizing  using  the  manikin 
and  the  radiography  equipment  in  the  laboratory.  The  SME  and/or  instructor  can  randomly  select  what 
part  or  all  of  the  procedure  is  to  be  performed  in  order  to  counteract  any  information  passed  on  by  those 
who  have  been  tested  to  those  who  have  not  that  may  indicate  what  section  was  being  tested.  The 
laboratory  simulation  will  be  used  to  evaluate  performance  (Appendix  B).  The  observation  instrument  to 
be  used  to  allow  an  SME  to  reliably  and  validly  evaluate  the  performance  of  the  students  as  they  perform 
the  skills  in  the  laboratory  will  be  the  same  whether  the  student  is  in  group  1  or  2. 


In  addition  to  the  performance  measures,  four  questionnaires  will  be  administered  (see  Appendix  C):  1)  a 
background  questionnaire,  which  will  be  administered  at  the  start  of  the  study;  2)  a  questionnaire 
assessing  the  students’  attitudes  and  beliefs  about  the  legacy  training  environment,  which  will  be 
administered  after  the  students  complete  the  component  of  the  course  in  which  they  experience  the  legacy 
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system;  3)  a  questionnaire  assessing  the  students’  attitudes  and  beliefs  about  the  simulation  software  and 
the  PACS,  which  will  be  completed  after  that  segment  of  the  course  in  which  the  students  experience  the 
simulation  system,  and;  4)  an  end  of  the  study  questionnaire  comparing  the  two  training  approaches. 

Data  Collection  and  Statistics 
Sample  size 

Subjects  will  be  recruited  from  the  Radiography  109  laboratory  classes  scheduled  for  the  spring 
Semester  in  2009.  All  students  attending  these  labs  will  be  invited  to  participate.  We  estimate 
that  the  sample  of  30-40  students  will  yield  moderate  effect  sizes  (0. 4-0.6)  and  will  have 
moderate  power  (0.6-0.7). 

Statistical  Methods 

Performance  of  the  radiology  simulation  training  condition  will  be  statistically  compared  to  that 
of  the  legacy  training.  Two  performance  measures  will  be  computed  for  each  condition  (legacy, 
simulation).  The  first  will  be  derived  from  the  written  exam  by  summing  the  number  of  correct 
answers.  (In  addition,  subscores  may  be  obtained  for  positioning  and  interpretation.)  The  second 
will  be  based  on  the  ratings  of  student  performance  in  the  laboratory  simulation.  If  personnel 
allows,  we  strive  to  have  two  SMEs  rate  each  student,  the  two  independently  completed  observer 
instruments  for  each  student  will  be  compared  and  large  differences  (more  than  2  scale  point 
difference)  in  task  ratings  will  be  adjudicated.  The  reliability  of  raters  in  terms  of  coefficient 
alpha  will  be  computed  for  each  pair  of  raters  across  common  students.  The  rating  instruments 
will  be  summed  across  tasks  to  arrive  at  a  total  rating  or  score.  The  two  total  rating  scores  for 
each  student  will  be  averaged  to  produce  the  second  performance  measure. 

The  two  performance  measures  will  each  be  subjected  to  a  within-subjects  ANOVA.  Thus, 
performance  means  derived  from  the  traditional  (legacy)  training  condition  will  be  contrasted 
with  means  derived  from  the  experimental  (simulation)  training  condition.  A  main  effect 
indicating  that  the  simulation  condition  means  are  significantly  larger  than  the  traditional  (legacy) 
condition  means  will  provide  conformation  of  the  hypothesis.  Effect  sizes  and  confidence 
intervals  will  also  be  computed. 

We  will  carry  out  a  similar  analysis  for  the  self-report  attitudinal  scales.  A  main  effect  will  show 
more  positive  attitudes  for  the  experimental  training  condition. 


Data  Analysis  Overview  : 

The  purpose  of  this  study  was  to  compare  the  effectiveness  and  acceptance  of  two  alternative  approaches 
for  teaching  introductory  radiology  students  about  anatomy,  positioning,  and  imaging.  In  the  traditional 
laboratory  approach  used  at  Mt.  Aloysius  for  teaching  radiographic  technology,  the  instructor 
demonstrates  positioning  and  imagery  techniques  using  a  manikin  and  discusses  existing  radiographic 
images  in  terms  of  anatomy,  positioning,  and  exposure.  During  the  laboratory  period,  the  students  are 
able  to  practice  positioning  on  the  manikin  and  with  each  other.  Although  they  can  position  the 
radiographic  equipment,  they  do  not  actually  take  images.  In  the  PACS/Sim  approach,  in  addition  to  the 
archived  digital  images,  the  students  use  the  computer-based  software  to  position  a  simulated  patient,  and 
to  take  and  critique  radiographic  images.  The  software  is  available  to  the  students  at  any  time,  so  they  are 
able  to  work  individually  at  their  own  pace  and  time.  We  hypothesized  that  the  simulation/PACS-based 
training,  which  allows  students  to  take  and  see  their  own  images,  and  to  repeat  that  process  as  many  times 
as  they  want,  would  be  more  effective  than  the  traditional  (legacy)  training  for  both  patient  positioning 
and  film  interpretation. 
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Method 


Participants 

All  students  enrolled  in  “radiology  101”  were  eligible  to  participate  in  the  study,  and  all  [but  XX]  agreed 
to  participate.  The  initial  sample  was  comprised  of  31  participants  of  whom  nine  were  males  and  22 
females.  Seventy-four  percent  of  the  participants  were  in  the  18-23  year  age  range,  and  only  one  person 
was  older  than  33  years.  All  but  one  of  the  participants  were  first-year  students,  and  only  two  had  a 
previous  (associates)  degree. 

Design 

There  were  three  laboratory  sections  for  this  course.  Students  in  laboratoiy  sections  1  and  3  were 
assigned  to  Group  1 .  Students  in  laboratory  section  2  were  assigned  to  Group  2.  During  the  course,  the 
students  studied  radiographic  procedures  for  six  anatomical  areas  (or  topics):  Bony  thorax,  ribs,  scoliosis 
series;  cervical  and  thoracic  spine;  lumbar  spine,  sacrum,  coccyx;  skull  and  sinuses;  facial  and  nasal 
bones;  and  mandible,  temperomandibular  joints,  zygomatic  arches.  For  the  first  three  topics,  students  in 
Group  1  were  taught  imaging  procedures  and  film  interpretation  using  simulation  software  and  the  PACS. 
Group  2  students  were  instructed  in  procedures  and  interpretation  using  the  legacy  training  method.  For 
the  last  three  topics,  Group  1  students  were  taught  procedures  and  film  interpretation  using  the  legacy 
training  method  while  Group  2  students  were  taught  procedures  and  film  interpretation  using  the 
simulation  software  and  PACS. 

Dependent  Variables 

Four  performance  measures  were  assessed  at  the  completion  of  each  of  the  six  topic  areas.  Three  of  these 
(anatomy,  positioning,  and  imaging)  were  derived  from  a  written  test  developed  by  the  instructors.  The 
fourth  was  a  simulation  test  in  which  the  participants  were  required  to  conduct  a  (simulated)  radiological 
exam.  The  simulation  test,  which  was  performed  using  the  manikin  as  the  patient,  included  items 
assessing  the  positioning  of  the  patient,  use  of  the  equipment,  and  completeness  of  assessment. 

Because  the  number  of  items  on  each  component  of  the  written  test  (anatomy  positioning,  imaging) 
differed  across  the  topic  areas,  we  converted  the  scores  to  percent  correct.  The  a  simulation  test  was 
graded  by  the  instructor  using  a  specifically  designed  evaluation  instrument  comprised  of  10  items,  each 
of  which  was  graded  on  a  five-point  scale  ranging  from  fail  to  high  pass.  For  ease  of  comparison  we 
transformed  this  to  a  100  point  scale,  which  could  range  from  20  to  100. 

To  create  the  dependent  variables,  we  computed  the  mean  score  on  each  of  the  four  measures  for  the  three 
tests  taken  using  the  PACS/Sim  software  and  for  the  three  tests  taken  using  the  traditional  laboratory 
approach.  Thus,  for  each  participant,  we  had  a  total  of  four  dependent  measures  (anatomy,  positioning, 
imaging,  and  simulation)  derived  from  tests  taken  using  the  PACS/Sim  software  and  four  measures  taken 
under  the  traditional  laboratory  approach. 

We  developed  four  questionnaires  for  this  study:  a  background  questionnaire,  a  questionnaire  assessing 
the  students’  opinions  about  the  PACS/Sim  training,  a  questionnaire  assessing  the  students’  opinions 
about  the  traditional  (legacy)  training,  and  a  questionnaire  comparing  the  two  training  approaches. 

Procedure: 


At  the  outset  of  the  course,  the  students  completed  the  background  questionnaire.  Testing  for  procedures 
and  film  interpretation  on  each  of  the  six  topics  occurred  after  training  on  that  topic.  At  the  end  of  each 
phase  of  training  (i.e.,  after  training  on  the  third  topic,  when  the  two  groups  switched  training  method) 
and  at  the  end  of  the  course  (i.e.,  after  training  on  the  last  topic),  the  students  completed  the  questionnaire 
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assessing  their  views  about  the  effectiveness  of  the  training  methodology  they  had  just  used.  At  the  end 
of  the  course,  they  completed  the  questionnaire  in  which  they  were  asked  to  compare  the  two  approaches 
on  a  number  of  dimensions. 

Results 

Performance  Assessment 

We  used  a  within-subjects  analysis  to  compare  performance  under  the  PACS/Sim  and  traditional 
laboratory  approaches.  Figures  1  through  3  show  the  comparison  between  the  two  approaches  for 
the  three  components  of  the  written  test  -  anatomy,  positioning,  and  imaging.  There  were  no 
differences  in  the  mean  anatomy  or  positioning  scores  under  the  two  conditions,  but  the  mean 
imaging  score  under  the  traditional  laboratory  condition  was  significantly  higher  than  those  under 
the  PACS/Sim  condition  (F(l,29)  =  5.37,  P<.03,  eta  sq  =  .16).  The  mean  simulation  test  scores 
are  shown  in  Fig.  4.  Analysis  showed  a  marginally  significant  difference  between  the  two 
training  approaches  (F(l,29)  =  4.09,  P<.06,  eta  sq.  =  .12)  in  favor  of  the  traditional  laboratory 
condition. 


Figure  1.  Comparison  between  two  training  conditions  on  Anatomy  Score. 
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Figure  2.  Comparison  between  two  training  conditions  on  Positioning  Score 
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Figure  3.  Comparison  between  two  training  conditions  on  Imaging  Score 


Figure  4.  Comparison  between  two  training  conditions  on  Simulation  Score. 
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Attitudes  towards  the  Two  Training  Approaches 

Learning  mode  preference 

In  the  background  questionnaire  we  asked  the  participants  to  rank  their  preferences  for  five  alternative 
learning  modes:  listening,  visualizing,  discussing,  doing,  and  reading.  The  minimum,  maximum  and 
mean  ranking  for  each  of  the  five  modes  are  shown  in  Table  1,  with  1  being  the  most  preferred  and  5  the 
least  preferred  mode.  As  the  table  indicates,  the  participants  have  a  clear  preference  for  learning  by 
doing.  The  mean  ranking  was  1.47;  two-thirds  of  the  participants  ranked  this  as  their  most  preferred 
mode.  Learning  by  reading  is  clearly  the  least  preferred  alternative:  The  mean  ranking  was  4.29;  two- 
thirds  ranked  learning  by  reading  as  their  least  preferred  mode. 


Table  1.  Participants’ preferences  for  learning  mode 


mode 

Minimum  rank 

Maximum  rank 

Mean  rank 

Std  deviation 

Listening 

2 

5 

3.63 

0.93 

Visualizing 

1 

5 

2.34 

1.18 

Discussing 

1 

4 

3.18 

0.80 

Doing 

1 

4 

1.47 

0.81 

Reading 

1 

5 

4.29 

1.27 

There  were  no  differences  between  the  participants  who  started  with  PACS/Sim  and  those  who  started 
using  the  traditional  laboratory  on  their  learning  preferences  or  on  any  of  the  background  questions. 


Participants  '  assessment  of  training  approaches 


At  the  midpoint  and  end  of  the  course,  the  students  completed  a  questionnaire  asking  their  opinions  about 
the  training  approach  they  had  just  experienced.  Using  a  similar  within-subjects  analysis  as  was  done  for 
the  performance  scores,  we  compared  the  participants’  assessment  of  the  PACS/Sim  to  the  traditional 
laboratory  condition.  Students  responded  to  the  items  on  each  questionnaire  using  a  5-point  Likert  type 
scale,  where  1  reflected  a  negative  rating  and  5  a  positive  rating.  The  items  were  grouped  into  four 
categories:  training  quality,  usefulness  of  the  laboratory  period,  radiological  skill  training  effectiveness, 
and  overall  evaluation  of  the  training.  The  mean  scores  for  these  four  categories  are  shown  in  Figures  5, 
6,  7,  and  8,  respectively.  As  the  figures  clearly  show,  there  was  a  strong  preference  for  the  traditional 
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laboratory  approach  over  the  PACS/Sim  approach.  In  all  cases,  the  mean  rating  for  the  traditional 
laboratory  approach  was  significantly  higher  than  that  for  the  PACS/Sim  approach. 
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Figure  5.  Comparison  between  two  training  conditions  on  Participants’  evaluation  of  quality  of  training, 

F(  1 , 28  )  =  86. 1 8,  p<.0 1 ,  effect  size  =  76 


Figure  6.  Comparison  between  two  training  conditions  on  Participants’  evaluation  of  laboratory  time,  F(l, 

28)  =  57.54,  p  <.01,  effect  size  =  67 
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Figure  7.  Comparison  between  two  training  conditions  on  Participants’  evaluation  of  radiographic  skills, 

F(l,28)  =  35.21,  p  <.01,  effect  size  =.56 
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Figure  8.  Comparison  between  two  training  conditions  on  participants’  overall  evaluation,  F(l,  28)  = 

86.17,  p  <.01,  effect  size  =.78 


The  end  of  study  questionnaire  asked  the  students  to  compare  the  two  approaches  on  a  number  of 
dimensions.  We  coded  the  response  alternatives  from  -5  (strong  preference  for  PACS/Sim)  to  +5  (strong 
preference  for  traditional  laboratory),  with  0  indicating  no  preference.  As  shown  in  Table  2  every  mean 
was  positive  indicating  a  mild  to  strong  preference  for  the  laboratory  approach.  The  preference  for  the 
traditional  laboratory  was  particularly  strong  on  positioning  and  practicing.  For  only  two  items, 
recognizing  incomplete  views  in  the  radiographic  images  and  evaluating  films,  was  the  students’ 
preference  close  to  neutral  (0). 
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Table  2.  Relative  comparison  of  two  training  approaches 


Measure 

Mean 

Std  error/std 
dev 

Overall  preference 

4.31 

0.14  0.71 

Patient  positioning 

4.60 

0.09  0.50 

Select  appropriate  view 

2.83 

0.41  2.26 

Recognize  incomplete  views 

1.43 

0.55  3.04 

Position  in  clinical  setting 

4.60 

0.10  0.56 

Evaluating  films  in  clinical 
setting 

0.53 

0.68  3.71 

Increasing  confidence  in 
performing  procedures 
competently 

2.52 

0.53  1.94 

Most  effective  feedback 

4.00 

0.33  2.87 

Best  opportunities  to  practice 

4.40 

0.20  1.80 

Efficient/effective  use  of  time 

3.33 

0.21  1.13 

In  the  background  questionnaire  that  the  participants  completed  at  the  outset  of  the  study,  we  asked  the 
participants  two  questions  about  computers  and  technology:  how  they  would  rate  their  computer  skills, 
and  how  comfortable  they  are  trying  and  using  new  technologies.  We  hypothesized  that  those  who  said 
they  were  more  proficient  with  computers  or  those  who  embraced  new  technologies  would  be  more 
accepting  of  the  PACS/Sim  approach.  We  found  no  evidence  for  a  negative  biased  towards  computer 
technology.  The  majority  of  the  students  rated  themselves  as  having  good  or  very  good  computer  skills, 
and  most  students  claimed  they  were  open  to  new  technologies.  Within  the  restricted  range  of  ratings,  we 
found  no  relationship  between  the  participants’  ratings  of  their  computer  skills  and  their  evaluation  of  the 
alternative  approaches  or  their  comfort  with  new  technology. 

The  students’  responses  to  several  open-ended  questions  included  in  the  post-study  questionnaire 
illuminate  their  preference  for  the  traditional  laboratory.  One  comment  voiced  by  a  number  of  students 
was  that  they  spent  too  much  time  learning  how  to  use  the  simulation  software.  Another  comment  was 
that  the  software  did  not  provide  sufficient  flexibility  in  positioning  the  patient.  But  overwhelmingly,  the 
most  often  voiced  comment  was  that  the  students  wanted  hands-on  practice  on  a  real  patient  (each  other) 
or  on  a  manikin  that  they  could  directly  manipulate  and  not  on  a  patient  image  that  they  manipulated  via 
pushing  buttons  on  a  computer  keyboard.  This  is  consistent  with  their  preferences  for  learning  by  doing 
(see  Table  1). 

Discussion 

Contrary  to  our  hypothesis  that  the  PACS/Sim  training  would  be  superior  to  the  traditional  training,  we 
found  no  difference  in  the  performance  on  anatomy  or  positioning  under  the  two  conditions,  and  a 
significant  difference  in  favor  of  the  traditional  laboratory  on  imaging.  In  addition,  their  performance  on 
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the  simulation  test,  which  emphasizes  positioning,  was  also  stronger  in  the  traditional  laboratory 
condition.  Similarly,  although  we  hypothesized  that  the  PACS/Sim  software  would  provide  the  students 
with  more  opportunity  to  practice  positioning  the  patient,  the  students  strongly  preferred  to  practice  on 
themselves  or  a  manikin.  It  is  possible  that  this  marginally  significant  difference  in  favor  of  the 
laboratory  training  on  the  simulation  test  can  be  explained  by  the  fact  that  the  test  was  performed  on  the 
manikin,  and  thus  those  who  were  trained  and  tested  using  the  same  simulation  method  were  at  an 
advantage. 

We  hypothesized  that  the  PACS/Sim  software  would  offer  the  students  a  richer  opportunity  for  learning 
about  imaging  because  it  allows  the  students  to  take  and  then  examine  an  image  to  see  whether  it  captured 
the  critical  anatomical  features  they  were  after,  and  to  do  this  repeatedly  until  they  were  satisfied  that  they 
had  a  suitable  image.  Contrary  to  our  expectation,  the  results  revealed  that  the  students  actually 
performed  better  with  the  traditional  approach  in  which  the  instructor  showed  and  discussed  with  them 
stored  images  taken  by  someone  other  than  themselves.  One  possible  explanation  is  that  the  students 
learned  more  from  a  rich  and  critical  examination  of  existing  images  than  they  w  ere  able  to  derive  from 
the  limited  feedback  provided  by  the  computer  (was  there  any??)  about  their  own  images.  Even  though 
they  were  able  to  see  the  images  they  took,  they  may  yet  be  able  to  see  the  flaws  and  limitations  in  the 
image  without  help  from  the  instructor  or  a  more  experienced  individual. 

At  the  outset  of  the  study  participants  said  that  they  most  prefer  learning  by  doing,  and  that  preference 
was  consistent  with  their  positive  evaluation  of  the  traditional  laboratory  and  was  voiced  frequently  in 
their  responses  to  open-ended  questions  about  the  two  approaches.  Given  that  the  students  felt  competent 
in  using  computers  and  were  open  to  new  technology,  we  cannot  attribute  their  negative  evaluations  of 
the  PACS/Sim  approach  to  an  anti-technology  bias. 

Limitations, 

This  study  was  conducted  the  first  time  the  PACS/Sim  technology  was  used  for  instruction  at  Mt. 
Aloysius,  so  the  results  do  not  provide  a  definitive  comparison  of  the  two  approaches.  Future  students 
may  benefit  from  improvements  in  use  of  the  computer  technology  that  instructors  can  make  based  on 
their  initial  experience,  leading  to  a  different  pattern  of  results.  A  second  limitation  is  that  because  the 
study  was  conducted  as  part  of  classroom  training,  we  were  not  able  to  counter-balance  the  two 
treatments  appropriately  or  randomly  assign  participants  to  treatments. 

Conclusions 

This  study  evaluated  a  first  attempt  to  use  computer  simulation-based  technology  for  teaching 
radiography.  We  found  that  the  participants  clearly  preferred  the  traditional  laboratory  experience  over 
the  PACS/Sim  experience  and  the  performance  results  showed  some  support  for  that  preference. 
Instructors  can  use  experiences  derived  from  this  first  administration  of  the  class  using  the  PACS/Sim 
technology  to  improve  the  effectiveness  and  acceptance  of  the  technology.  The  students’  reported 
difficulty  in  manipulating  the  computer-based  patient  suggests  that  more  initial  training  in  using  the 
software  is  needed.  In  the  future  it  may  also  be  useful  to  evaluate  the  effectiveness  and  acceptability  of  an 
approach  that  integrated  the  two  training  methodologies. 
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Simulation  Study  #  3 


Study  Results 

A  Comparison  of  Using  High  Fidelity  Simulation  vs.  Virtual  Case  Studies  in  the  Acquisition  of 

Complex  Clinical  Nursing  Skills 


Background 


In  nursing  education,  simulation  is  an  attempt  to  ‘mirror’  or  replicate  aspects  of  a  clinical  situation  so  that 
the  situation  can  be  better  managed  when  it  occurs  in  a  real  clinical  environment.  The  word  fidelity  is 
often  used  in  simulation  and  is  defined  as  “precision  of  reproduction.”  Simulation  attempts  to  achieve  a 
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high  enough  fidelity  to  convince  the  learners  that  they  are  using  something  that  resembles  what  they 
would  encounter  in  real  life  (1 1).  Simulations  range  from  the  use  of  case  studies  to  educate  students 
about  patient  situations  or  the  use  of  role-playing  to  engage  students  in  a  particular  clinical  situation.  Part 
task-trainers  and  low-fidelity  simulators  are  used  to  help  students  learn  and  practice  particular  clinical 
skills  and  intervene  with  basic  changes  in  patient  condition.  High-fidelity  patient  simulators  are 
sophisticated  computer  simulated  manikins  that  provide  a  high  level  of  interactivity  and  realism  for  the 
learner  (6).  Computer-based  scenarios  enable  students  to  participate  in  a  virtual  clinical  environment 
where  they  focus  on  assessment,  problem  solving,  skill  performance,  and  decision  making  during  the  care 
of  a  virtual  patient. 

High-fidelity  simulation  experiences  represent  a  teaching  strategy  in  which  students  develop  and  practice 
clinical  decision-making  and  skills  on  highly  sophisticated,  realistic  manikins.  A  major  advantage  of  this 
strategy  is  that  it  encompasses  the  three  domains  of  learning:  cognitive,  psychomotor  and  affective  (9). 
The  use  of  simulation  provides  value  to  nursing  educators  because  of  it  allows  students  to  develop 
interactive  critical  thinking  skills  and  assessment  abilities  in  a  non-threatening  environment  (10).  The  use 
of  complex  scenarios  during  the  simulation  activity  allows  students  to  assess  changing  medical  conditions 
and  identify  critical  incidents.  The  measurement  of  physical  parameters  and  communication  with  the 
simulated  patient  enable  students  to  determine  appropriate  interventions  and  evaluate  the  need  for  further 
intervention  (6). 

In  contrast,  simulation  using  computer-based  virtual  clinical  environments  is  an  innovative  teaching 
strategy  that  enables  students  to  engage  in  a  virtual  hospital  setting  where  they  interact  with  virtual 
patients,  and  access  and  evaluate  realistic  information  essential  for  patient  care.  The  virtual  activity 
allows  students  to  make  decisions  and  set  priorities  within  the  virtual  hospital  setting.  Students  can  also 
develop  effective  communication,  documentation,  assessment  and  critical  thinking  skills. 

Few  studies  have  been  published  that  examine  what  effect  simulation  activities  have  on  the  attainment 
and  retention  of  knowledge  and  on  performance  on  written  exams  that  assess  knowledge  and  the 
application  of  critical  thinking  skills.  Of  those  reported,  Katz  et  al  (8)  found  no  different  between  a  group 
exposed  to  simulation  training  and  those  not  exposed  to  training  on  clinical  instructor  ratings  of 
competence  in  the  clinical  setting  or  on  the  participants’  self-assessment  of  clinical  judgment.  On  the 
other  hand,  Childs  and  Sepples  (3)  showed  that  students  retain  knowledge  learned  from  a  simulation  for  a 
longer  period  of  time  compared  to  the  traditional  method  of  teaching  (3),  where  knowledge  is  assessed 
through  the  use  of  verbal  questioning  or  written  tests. 

Even  fewer  studies  have  contrasted  the  effectiveness  of  differing  types  of  simulation  activities  on 
knowledge  or  performance.  Howard  (4)  compared  the  use  of  a  human  patient  simulator  (HPS)  and  an 
interactive  case  study  approach  for  the  acquisition  of  knowledge  and  critical  thinking.  A  Health 
Education  System,  Inc  (HES1)  exam  (see 

http://wAvw.associatedcontent.com/article/35420/hesi  exams  health  education  svstems.html?cat=4)  was 

used  to  measure  knowledge  and  critical  thinking  ability.  Analysis  revealed  a  significant  difference  with 
respect  to  knowledge  gained  and  a  marginally  significant  (p=.051)  difference  on  critical  thinking  ability 
on  the  post -test,  with  the  group  using  the  HPS  scoring  higher  on  the  post  test  than  the  group  that  used  the 
interactive  case  study.  Johnson  et  al.  (7)  compared  a  HPS  to  a  computer-assisted  instruction  (CD-ROM) 
approach  and  a  control  group  in  terms  of  scores  on  lower-and  higher-level  cognition,  critical  thinking,  and 
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performance  standards.  Critical  thinking  was  defined  as  the  participants’  ability  to  assess,  intervene,  and 
evaluate  care  of  chemical  casualty  patients,  and  performance  was  defined  as  the  ability  to  assess  and 
implement  appropriate  care.  Results  showed  that  participants  in  the  HPS  condition  performed 
significantly  better  on  the  higher  level  cognition,  critical  thinking,  and  performance  evaluation 
instruments  compared  to  the  CD-ROM  and  control  groups. 

Further  research  is  needed  to  ascertain  the  effects  of  varying  types  of  simulation  activities  on  the 
acquisition  and  application  of  knowledge  as  assessed  by  written  exams.  As  was  done  in  the  previously 
cited  studies,  for  meaningful  evaluation  of  simulation-based  learning,  assessment  should  go  beyond 
factual  knowledge,  and  incorporate  measures  to  assess  the  student’s  ability  to  identify  critical  incidents, 
competence  with  skill  performance,  communication  effectiveness,  and  implementation  of  appropriate 
nursing  interventions.  (5)  The  National  Council  Licensure  Examination  (NCLEX)  incorporates 
applications  of  knowledge  to  multiple  categories  of  patient  needs,  including  effective  care  management. 

In  terms  of  Bloom’s  (1)  taxonomy,  NCLEX-type  questions  are  written  at  the  application  and  above  level. 

The  purpose  of  this  study  was  to  contrast  training  of  complex  clinical  knowledge  and  skills  augmented  by 
interactive  computer-based  clinical  exercises  with  training  augmented  by  mannequin  simulation-based 
clinical  exercises  to  see  which  method  results  in  higher  scores  on  a  NCLEX-type  examination  and  which 
method  was  preferred  by  the  students. 


Hypotheses 


Given  the  findings  of  Howard  et  al.  (4)  and  Johnson  et  al.  (7),  we  hypothesize  that  students  utilizing  the 
mannequin-based  simulation  will  have  higher  scores  on  an  NCLEX-type  exam  than  students  utilizing  the 
interactive  computer-based  virtual  simulation. 

We  have  found  that  nursing  students  express  a  preference  for  hands-on  training.  For  this  reason  wre 
hypothesize  that  students  will  prefer  the  mannequin  simulation  over  the  virtual  software  simulation  for  the 
acquisition  of  knowledge  and  application  to  a  complex  clinical  case.  Given  the  preference  for  hands-on 
training,  we  also  hypothesize  that  students  will  feel  better  prepared  for  the  NCLEX  and  for  application  of 
clinical  skills  to  patient  care  by  the  mannequin  than  by  the  virtual  software  simulate 
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Nursing  student  participant  using  manikin  based  simulation 


Research  Design 


Participants 

Forty-eight  nursing  students  enrolled  in  senior  seminar  classes  consented  to  participate  in  this  study. 


Independent  Variables 

The  first  of  two  independent  variables  was  type  of  training  augmentation  manipulated  over  two 
conditions.  The  first  condition,  interactive  computer-based  virtual  exercises  (VCE),  was  developed  from 
commercially  available  software.  The  second  condition,  mannequin-based  simulation  exercises  (MS), 
was  developed  using  a  commercially  available  high  fidelity  programmable  HPS.  The  computer-based 
virtual  clinical  scenarios  were  obtained  from  Evolve/Elsevier’s  Virtual  Clinical  Excursion  software 
package1;  the  simulated  activity  on  the  mannequin  was  replicated  from  the  Virtual  Clinical  Excursion 
software  program. 


1  Permission  was  obtained  from  the  publisher  to  use  the  software  program  in  this  study 
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The  second  independent  variable  was  medical  event  (or  topic),  varied  over  two  events.  The  events  used 
in  this  study,  which  were  selected  from  the  Virtual  Clinical  Excursion  software  package,  included 
Perioperative  Care  and  DVT/  Pulmonary  Embolism.  These  events  are  typically  covered  in  upper  level 
seminar/clinical  classes,  and  were  judged  to  be  of  equal  complexity. 


Procedure 

Participants  were  assigned  to  training  augmentation  condition  based  on  the  laboratory  section  in  which 
they  enrolled.  For  the  first  event  (perioperative  care),  the  students  in  one  lab  section  were  assigned  to  the 
VCE  condition  and  the  students  in  the  other  lab  section  were  assigned  to  the  MS  condition.  All 
participants  completed  training  on  that  medical  event.  For  the  second  event  (DVT/Pulmonary 
Embolism),  participants  switched  training  augmentation  conditions.  Those  initially  in  the  VCE  condition 
switched  to  the  MS  condition  and  those  initially  in  the  MS  condition  switched  to  the  VCE  condition.  All 
students  completed  training  on  the  second  medical  event.  At  the  conclusion  of  each  event  all  students 
completed  an  NCLEX-type  multiple  choice  exam  designed  to  assess  knowledge  and  application  skills  for 
that  event,  and  an  attitude  questionnaire  designed  to  assess  participants’  satisfaction  with  the  training 
augmentation  they  experienced.  Following  the  completion  of  the  second  nursing  event,  participants 
completed  a  questionnaire  comparing  their  attitudes  and  preferences  for  the  two  training  augmentation 
conditions. 


Results 


Performance  Assessment 


The  NCLEX-type  multiple  choice  exam  scores  for  participants  under  the  two  training  procedures  were 
compared  in  a  within-subjects  ANOVA.  In  this  comparison,  the  participant’s  performance  under  the  MS 
training  augmentation  condition  was  compared  to  his/her  performance  under  the  VCE  condition.  Based 
on  prior  research  (4,  7),  we  hypothesized  that  performance  using  the  MS  would  be  significantly  higher 
than  performance  under  the  VCE.  As  shown  in  Figure  1,  however,  participants  attained  significantly 
higher  test  scores  under  the  VCE  than  MS  training  procedures,  F(l,  39)  =  1 1 .27,  p  <  0.01. 

We  also  compared  the  training  effectiveness  for  each  of  the  two  medical  events  used  in  the  training 
(Perioperative  Care  and  DVT/  Pulmonary  Embolism)  as  between-subjects  analyses.  The  results  are  shown 
in  Figures  2  and  3.  For  both  medical  events  the  mean  test  score  was  higher  when  the  training  procedure 
was  VCE  than  when  it  was  MS,  but  only  the  difference  for  the  DVT/  Pulmonary  Embolism  Medical 
Event  was  significant,  F(l,  21)  =  1 1 .16,  p  <  0.01.  We  conclude  that  participants  benefited  more  from  the 
VCE  than  the  MS  training,  particularly  when  learning  how  to  deal  with  a  DVT/  Pulmonary  Embolism 
Medical  Event. 


65 


VCE  SIM 


Training  Condition 


Figure  1 .  Mean  Test  Scores  for  Participants  in  the  VCE  and  MS  Training  Conditions 


Figure  2.  Mean  Test  Scores  for  Participants  in  the  VCE  and  MS  Training  Conditions  for  the 

Perioperative  Care  Medical  Event 
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Figure  3.  Mean  Test  Scores  for  Participants  in  the  VCE  and  MS  Training  Conditions  for  the  DVT/ 

Pulmonary  Embolism  Medical  Event 


Participants’  Attitudes  towards  the  Two  Training  Approaches 


In  the  evaluation  conducted  at  the  end  of  each  training  session  participants  rated  the  training  type  received 
on  a  7-point  Likert-type  scale  on  which  1  represented  a  poor  rating  and  7  an  excellent  rating.  The  mean 
rating  score  for  VCE  training  condition  (3.88  )  was  not  significantly  different  (t<  1 .0)  from  the  mean 
rating  score  for  MS  training  condition  (4.08). 

The  End  of  Study  Questionnaire  included  five  pairs  of  items  comparing  the  two  training  approaches  in 
terms  of  learning  and  application.  Participants  responded  on  a  Likert-type  rating  scale  ranging  from  1 
(low  effectiveness/agreement)  to  7  (high  effectiveness/agreement).  Table  1  shows  the  mean  rating  scores 
and  standard  deviations  under  each  of  the  two  training  conditions  for  the  five  contrasts.  The  means  on  the 
first  four  contrasts  were  very  close  or  identical,  suggesting  no  preference  for  either  ty  pe  of  training. 

Paired  t-tests  performed  on  the  rating  scores  showed  that  only  contrast  5  (patient  evaluation)  produced  a 
significant  result,  t(  46)  =  3.48,  p  <  0.01.  Participants  felt  that  the  MS  training  helped  them  better 
evaluate  patients’  vital  signs  than  training  with  the  VCE.  This  result  was  the  only  support  for  the 
hypothesis  that  students  would  prefer  the  MS  training  for  the  application  of  their  clinical  skills  to  patient 
care.  The  participants’  preference  for  the  MS  training  over  the  VCE  training  contrasts  with  the 
performance  results,  which  were  in  favor  of  the  VCE  training 
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In  addition  to  the  items  addressing  specific  aspects  of  the  training,  the  End  of  Study  Questionnaire  also 
asked  the  participants  which  type  of  training  they  preferred  overall.  For  this  question,  they  responded  on 
a  7-point  scale  with  VCE  at  one  end  and  MS  at  the  other.  Responses  towards  the  lower  or  upper  end  of 
the  scale  would  indicate  a  preference  for  the  VCE  or  MS  training,  respectively.  Responses  in  the  middle 
of  the  scale  indicate  neutrality  in  terms  of  preference.  The  mean  response  was  4.36  (s.d.  =  1.83),  which  is 
not  significantly  different  from  4,  the  midpoint  on  the  scale. 

In  fact,  almost  all  of  the  ratings  shown  in  Table  1  are  not  significantly  different  from  the  midpoint  on  the 
scale.  The  exceptions  were  the  ratings  for  contrast  I  (extent  to  which  it  “felt  like  I  was  dealing  with  a  real 
patient”),  where  both  the  VCE  and  MS  ratings  were  significantly  lower  than  4  (p  <  .006  and  .p<  04, 
respectively),  and  the  VCE  rating  for  contrast  5  (effectiveness  of  the  software  in  helping  you  evaluate 
patient’s  vital  signs),  which  was  also  significantly  lower  than  4  (p<001).  Taken  as  a  whole,  the  ratings 
indicate  that  the  participants  were  neutral  about  both  training  approaches  without  strong  preference  for  the 
effectiveness/utility  of  either  one. 

Table  1.  The  Five  Item  Pairs  From  The  End  Of  Experiment  Questionnaire  Contrasting  Training  Type 


VCE  Item 

MS  ITEM 

VCE 

MS 

mean/sd 

mcan/sd 

Contrast  1 

When  I  used  the  Virtual 

When  I  used  the  manikin 

3.31/ 

3.54/ 

Clinical  Excursion  (VCE) 

simulation  (MS),  it  felt 

1.65 

1.50 

software,  it  felt  like  I  was 

like  I  was  dealing  with  a 

dealing  with  a  real  patient 

real  patient. 

Contrast  2 

I  prefer  to  use  the  virtual 

1  prefer  to  use  the 

4.00/ 

3.83/ 

excursion  software  (VCE) 

manikin  simulation  (MS) 

1.76 

1.63 

rather  than  the  manikin 

rather  than  the  virtual 

simulation  (MS)  as  a 

excursion  software 

means  to  better  apply 

(VCE)  as  a  means  to 

information  to  a  NCLEX- 

better  apply  information 

type  exam. 

to  a  NCLEX-type  exam. 

Contrast  3 

How  effective  was  the 

How  effective  was  the 

3.96/ 

3.96/ 

virtual  excursion  software 

manikin  simulation  (MS) 

1.37 

1.27 

(VCE)  in  helping  you  to 

in  helping  you  to  learn 

learn  the  course  material? 

the  course  material? 

Contrast  4 

How  useful  was  the 

How  useful  was  the 

3.74/ 

4.11/ 

virtual  excursion  software 

manikin  simulation  (MS) 

1.51 

1.32 

(VCE)  in  helping  you  to 

activity  in  helping  you  to 

apply  what  you  learned  in 

apply  what  you  learned 

class  to  a  simulated 

in  class  to  a  simulated 

patient? 

patient? 
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Contrast  5 

How  effective  was  the 

How  effective  was  the 

3.49/ 

4.21/ 

virtual  excursion  software 

manikin  simulation  (MS) 

(VCE)  in  helping  you 

in  helping  you  evaluate  a 

1.33 

1.47 

evaluate  a  patient’s  vital 
signs? 

patient’s  vital  signs? 

Discussion  and  Conclusions 


Contrary  to  our  hypothesis,  the  VCE  training  resulted  in  stronger  performance  on  the  knowledge  test  than 
the  MS  training.  Results  for  stronger  performance  on  the  VCE  training  may  relate  to  the  student’s  ability 
to  work  independently  and  privately  on  the  computer  without  distraction,  thus  enhancing  concentration 
ability.  Students  performing  the  MS  training  were  involved  in  actual  simulated  practice  with  an 
instructor  present.  Feelings  of  stress  induced  by  this  more  public  and  perhaps  implicitly  evaluative 
situation  could  have  affected  the  students’  performance  and  concentration,  and  ultimately  could  have 
resulted  in  the  lower  testing  outcome. 

In  general,  nursing  students  indicate  that  they  learn  from  ‘doing  ‘and  ‘hands-on’  practice.  Thus,  in  terms 
of  preference  and  perceived  learning  effectiveness  of  the  training,  our  hypotheses  that  participants  would 
prefer  the  MS  received  little  support.  Except  for  the  evaluation  of  patients’  vital  signs,  however,  there 
were  no  significant  differences  between  the  MS  and  VCE  ratings.  Further,  as  indicated  in  Table  1 ,  the 
mean  ratings  were  close  to  the  midpoint  for  both  the  MS  and  VCE  training,  indicating  no  strong 
preference  (i.e.,  mean  ratings  in  the  6-7  range)  for  either  type  of  training.  As  the  significantly  lower  than 
midpoint  rating  on  simulator  realism  (contrast  1)  suggest,  neither  the  MS  nor  the  VCE  simulation  gave 
the  students  the  feeling  that  they  were  interacting  with  a  real  patient.  Perhaps  instructors  need  to  give 
greater  attention  to  the  element  of  realism  in  simulated  training,  particularly  when  the  focus  is  on  complex 
clinical  skills  rather  than  individual  nursing  procedures,  since  even  in  the  MS  condition  students  did  not 
feel  that  they  were  dealing  with  a  real  patient. 

In  light  of  the  difference  between  the  results  of  previous  studies  contrasting  mannequin  with  virtual 
simulation  (4,  7)  and  the  present  one,  effectiveness  of  simulation  for  performance  on  multiple  choice 
exams  may  need  further  study.  Moreover,  the  significant  difference  in  performance  between  the  two 
training  approaches  for  the  DVT/Pulmonary  Embolism  medical  event,  but  not  the  Perioperative  Care 
medical  event,  suggests  that  additional  events  need  to  be  studied  in  order  to  understand  better  the  types  of 
events  for  which  the  VCE  is  more  effective. 


Limitations 


Because  this  study  was  conducted  within  the  context  of  a  course,  it  was  not  possible  to  randomly  assign 
students  to  condition.  Nor  was  it  possible  to  randomize  the  order  in  which  the  two  events  were  trained  or 
to  include  a  wider  range  of  complex  nursing  events. 
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Task  3)  Deliverable:  Evaluation  report  on  the  effectiveness  of  technology-delivered 

reinforcement  methods  following  Diabetes  Self-Management  Education  (DSME)  programs 


GOAL: 

The  goal  of  the  study  will  be  to  examine  the  effects  of  standard  methods  of  follow-up  by  Certified 
Diabetes  Educators  for  patients  with  diabetes  following  DSME  classes  versus  an  innovative  strategy 
utilizing  technology-delivered,  educational  pop  up  messages  via  the  Internet. 
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The  Effectiveness  of  Reinforcement  Methods  Following  Diabetes  Self-Management  Education 


Abstract 


There  is  considerable  interest  in  the  health  care  community  in  developing  new  strategies  that  are  effective 
in  enhancing  the  assimilation  and  retention  of  information  necessary  for  patient  self-management  of 
diabetes.  This  pilot  study  evaluated  the  effectiveness  of  computerized  follow-up  methods  compared  to 
the  standard  telephone  follow-up  in  adult  patients  with  Type  2  diabetes  who  have  participated  in  diabetes 
self  management  education  (DSME)  classes.  In  the  computerized  follow-up,  participants  accessed 
educational  and  motivational  messages  on  a  specially  designed  website  approximately  bimonthly  over  a 
nine-month  period  following  the  classes.  Each  message  included  informational  tips  about  a  particular 
topic  relevant  to  diabetes  control,  and  questions  about  the  participant’s  activities  relevant  to  that  topic. 

We  hypothesized  that  the  computerized  follow-up  would  result  in  improved  knowledge,  better  behavioral 
(health-related)  indicators,  and  greater  satisfaction  than  the  standard  follow-up  method,  a  telephone 
follow-up  call  three  months  after  the  classes. 


Analysis  of  the  data  show  ed  no  support  for  our  hypotheses.  There  was  an  increment  in  scores  on  the 
diabetes  knowledge  test  from  the  start  to  the  end  of  the  study,  but  the  improvement  was  significant  only  in 
the  telephone  group.  There  were  no  significant  differences  between  the  two  groups  in  behavioral 
indicators,  which  included  measures  of  A  1C,  cholesterol,  and  triglycerides.  Satisfaction  ratings  were  high 
in  both  groups,  but  not  significantly  different  between  the  groups.  Participants  in  the  computer  group 
reported  no  difficulty  in  accessing  and  using  the  website,  or  in  understanding  the  information  conveyed. 
They  found  the  follow-up  messages,  especially  those  about  medications,  useful,  but  this  did  not  result  in 
improved  behavioral  (health)  indicators  across  the  period  of  the  study.  We  conclude  that  the  computer- 
based  follow-up  needs  to  be  more  interactive  and  more  tailored  to  each  participant’s  individual  concerns 
and  goals. 


Introduction 


There  is  considerable  interest  in  the  health  care  community  in  developing  new  strategies  that  are  effective 
in  enhancing  the  assimilation  and  retention  of  information  necessary  for  patient  self-management  of 
diabetes.  This  pilot  study  compared  the  effectiveness  of  an  experimental  computerized  follow-up  method 
compared  to  the  standard  telephone  follow-up  method  in  adult  patients  with  Type  2  diabetes  who  have 
participated  in  DSME  classes  and  have  computer  internet  access.  The  classes  were  conducted  by  a 
diabetes  educator  at  the  Conemaugh  Diabetes  Institute  in  Johnstown,  PA.  They  addressed  topics 
important  to  individuals  with  diabetes  including  what  diabetes  is,  nutrition,  stress  management, 
managing  hyper/hypoglycemia,  monitoring  blood  sugar  level,  goal  setting,  and  behavior  change 
strategies.  The  computerized  follow-up,  which  was  accessed  on  the  web,  involved  computer-generated 
educational  messages,  resource  recommendations,  and/or  other  learning  tools  focusing  on  the  seven  self- 


71 


care  behaviors  advocated  by  the  American  Diabetes  Association  (ADA)  and  the  American  Association  of 
Diabetes  Educators  (AADE).  Educational  modules  for  the  purpose  of  reinforcing  the  information 
presented  in  the  DSME  classes  will  be  provided.  The  standard  telephone  follow-up  is  a  call  to  the  patient 
from  a  nurse  (or  CDE?)  three  months  after  the  classes  are  completed. 


Method 


Sample: 

Participation  was  solicited  among  adults  who  attended  the  diabetes  education  classes  at  Conemaugh 
Diabetes  Institute.  The  requirements  for  participation  were  diagnosis  of  type  II  diabetes  and  access  to  a 
computer  internet  access.  The  sample  included  8  males  and  13  females.  The  age  of  the  participants 
ranged  from  37  to  85,  with  a  mean  age  of  58.  Of  the  21  participants,  one  completed  grade  school,  1 1 
completed  high  school,  and  9  completed  college.  There  were  no  significant  differences  between  the 
telephone  and  computer  groups  in  these  variables. 


Follow-up  Intervention: 

Control-group  participants  received  standard  follow-up  by  a  CDE  that  consists  of  a  telephone  call  three 
months  after  the  classes  ended  to  discuss  their  progress/problems  and  answer  questions  relating  to  goals 
established  during  DSME.  Intervention-group  participants  were  instructed  to  access  a  learning  module  at 
a  website  developed  for  this  purpose  on  a  weekly  basis  for  three  months  following  enrollment,  then  bi¬ 
monthly  for  the  duration  of  the  study.  Computer-group  participants  received  no  telephone  follow-up  by  a 
CDE. 


Procedure: 

Following  enrollment,  but  prior  to  the  conduct  of  the  DSME  classes,  participants  completed  a  background 
questionnaire  and  a  diabetes  knowledge  test.  A  nurse  collected  clinical  information,  including  weight, 
blood  pressure,  cholesterol,  triglyceride,  and  A  1C  measures.  The  participants  then  attended  the  five  (?) 
DSME  classes,  and  met  individually  with  a  certified  diabetes  educator  (CDE)  to  establish  personal  goals 
for  better  managing  their  condition  (e.g.,  weight  loss,  exercise,  diet).  Following  the  classes,  the 
participants  in  the  computer-group  started  receiving  weekly  emails  instructing  them  to  access  a  learning 
module  on  the  web.  After  a  3  month  interval,  a  nurse  collected  A  1C  readings  from  all  participants.  At 
about  this  time,  the  participants  in  the  telephone  group  were  contacted  by  a  nurse  via  telephone.  After  3 
months,  the  computer  group  participants  received  bi-weekly  email  reminders  to  access  the  website  for  the 
next  learning  module.  At  six  months,  a  nurse  collected  clinical  data  including  A  1C,  triglyceride,  and 
cholesterol  readings.  At  nine  months,  the  end  of  the  study,  all  participants  completed  a  diabetes 
knowledge  test  and  a  satisfaction  survey.  A  nurse  collected  A1C  readings. 


Results 

We  examined  three  factors:  knowledge  gain,  behavioral  (clinical)  outcomes,  and  satisfaction  with  the 
program. 


72 


Knowledge  gain: 


Both  the  telephone  and  the  computer  groups  received  the  educational  classes,  and  there  was  a  significant 
gain  in  pre-  to  post-knowledge  scores.  In  examining  the  two  groups  separately,  we  found  that  the  gain  in 
the  telephone  group  was  statistically  significant  (t=4.61,  df-7,  p=.002),  but  the  gain  in  the  computer 
group  was  not  (t=1.04,  df=7,  ns). 


Figure  1.  Pre-  and  post-study  diabetes  knowledge  scores  by  experiment  condition. 


Clinical  (behavioral)  outcomes: 

A  1C  was  measures  at  four  points:  study  outset,  and  at  3,  6,  and  9  months.  As  shown  in  Figure  2,  in  both 
groups  there  was  a  significant  (F(l ,  1 9)  =  13.33,  p=.002)  drop  in  A1C  levels  during  the  first  3  months  of 
the  study,  and  then  a  slow  (nonsignificant)  rise.  There  was  no  statistically  significant  difference  in  scores 
between  the  two  groups. 
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■  Telephone 

■  Computer 


Figure  2.  A  1C  levels 


HDL  and  LDL  cholesterol  readings  were  taken  at  study  outset  an  at  the  6  month  point  in  the  study  (see 
Figures  3  and  4).  HDL  went  up  significant  in  the  computer  group  from  study  outset  to  six  months 
(t=2.85,  df=9,  p=  019),  but  not  in  the  telephone  group.  LDL  fell  slightly  in  the  telephone  group  and,  as 
HDL,  rose  significantly  in  computer  group. 


Figure  3.  HDL  scores  at  baseline  and  six  months  by  experiment  condition 
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Figure  4.  LDL  scores  at  baseline  and  six  months  by  experiment  condition 


The  triglyceride  measures,  which  were  taken  at  baseline  and  6  months,  are  shown  in  Figure  5.  The  means 
in  both  groups  dropped  from  baseline  to  six  months,  but  insignificantly  so.  Although  the  computer 
group’s  measures  are  lower  than  those  in  the  telephone  group,  the  difference  is  not  statistically 
significant. 
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Satisfaction: 


The  satisfaction  questionnaire  administered  at  the  end  of  the  study  addressed  the  participants’  satisfaction 
with  the  classes  and  the  follow-up  methodology,  and  their  assessment  of  the  value  of  the  follow-up 
approach.  Responses  were  made  on  a  7-point  scale.  On  satisfaction  items,  the  higher  the  response,  the 
more  positive  the  satisfaction. 


Overall,  as  shown  in  Table  1,  the  responses  to  the  program  were  above  the  midpoint  rating  in  both  groups. 
Only  one  of  the  1 1  items  addressing  these  factors  showed  a  significant  difference  between  the  control  and 
intervention  group.  Interestingly,  it  was  an  item  asking  how  concerned  they  were  about  privacy  issues 
when  using  a  computer  to  communicate  with  a  practitioner.  Those  who  experienced  the  computer  follow¬ 
up  were  significantly  less  concerned  than  those  who  received  the  telephone  follow-up,  suggesting  the 
computer  users  were  comfortable  with  that  mode  of  communication.  Although  only  this  item  yielded  a 
significant  difference,  we  noted  that  in  the  six  questions  addressing  the  convenience  (Q5_3) ,  usefulness 
(Q7_5,  Q9_6,  Q10J7),  and  overall  satisfaction  (Q2CM6,  Q21_17)  with  the  follow-up  methodology,  the 
mean  rating  was  higher  in  the  computer  group  than  the  telephone  group  in  four  of  the  items  and 
approximately  the  same  (within  .  1  points)  in  the  other  two,  offering  some  suggestive  evidence  that  those 
in  the  intervention  group  were  more  satisfied  with  the  follow-up  program  than  those  in  the  control  group. 


Condition 

N 

Mean 

Std. 

Deviation 

Std. 

Error 

Mean 

Q5 _ 3:  This  method  of  diabetes  follow-up  was 

[very  inconvenient/convenient]: 

telephone 

11 

4.6364 

2.41962 

.72954 

computer 

9 

6.0000 

1.41421 

.47140 

Q7_5:  How  useful  were  the  messages  for  you? 

telephone 

10 

5.1000 

2.42441 

.76667 

computer 

9 

5.6667 

1.80278 

.60093 

Q9_6:  To  what  extent  did  this  method  of 
follow'- up  help  you  to  manage  your  diabetes 

telephone 

11 

4.7273 

2.57258 

.77566 

computer 

9 

4.6667 

2.39792 

.79931 

Q10_7:  To  what  extent  did  this  method  of 
follow-up  encourage  you  to  meet  the  goals  you 

telephone 

11 

4.6364 

2.50091 

.75405 

established? 

computer 

9 

5.0000 

2.39792 

.79931 

Q1 2a_8a:  To  what  extent  are  you  concerned 
about  privacy  issues  when  speaking  to  a 

telephone 

10 

4.5000 

2.59272 

.81989 

practitioner  on  the  phone? 

computer 

9 

5.7778 

1.39443 

.46481 

Q12b_8b:  To  what  extent  are  you  concerned 
about  privacy  issues  when  using  the  computer 

telephone 

10 

3.5000 

2.59272 

.81989 

for  this  purpose? 

computer 

9 

6.0000 

1.50000 

.50000 

Q 1 3a_9a:  To  what  extent  do  you  prefer  to  talk 
with  a  diabetes  counselor  to  receive  information 

telephone 

11 

4.1818 

2.67650 

.80699 

computer 

9 

5.3333 

.86603 

.28868 

Q1 3b_9b:  To  what  extent  do  you  prefer  to 
receive  computer-  generated  messaging  to 

telephone 

10 

4.0000 

2.26078 

.71492 

high  =  not  concerned 


t=2.53,  p<.003 

NOTE:  this  scale  was  not  reversed; 


l.e. 


76 


receive  information? 

computer 

9 

5.0000 

2.12132 

.70711 

Q1410:  At  times  during  the  follow-up  program 

I  felt  the  need  to  talk  with  a  diabetes  counselor 

telephone 

10 

3.9000 

2.07900 

.65744 

computer 

9 

4.6667 

1.73205 

.57735 

Q20_16:  Overall,  how  satisfied  were  you  with 
this  method  of  education  follow-up? 

telephone 

11 

4.4545 

2.58316 

.77885 

computer 

9 

5.4444 

2.29734 

.76578 

Q21_17:  I  would  recommend  this  program  to 
other  persons  with  diabetes. 

telephone 

11 

5.5455 

1.75292 

.52853 

computer 

9 

5.4444 

2.55495 

.85165 

NOTE:  this  scale  was  not  reversed; 


Table  1.  Comparison  of  telephone  and  compute  groups  on  comparable  items  on  the  Satisfaction 
Questionnaire  [NOTE:  Q  numbers  in  column  1  of  the  table  (e.g.,  Q5_3)  refer  to  the  question  number  on  the 
computer  and  telephone  group  versions,  respectively.] 


I.e. 


Items  on  the  satisfaction  survey  pertinent  only  in  the  computer  group  addressed  the  helpfulness  of  the 
email  message  reminding  them  to  access  the  website,  their  difficulty  in  accessing  the  site  and  use  of  the 
site,  and  the  ease  of  understanding  the  informational  messages.  Responses  to  these  items  were  highly 
positive;  the  means  ranged  from  5.8  to  6.8,  and  the  median  and  modal  responses  were  7  on  all  of  them. 
We  also  asked  the  participants  in  the  computer  group  how  useful  specific  types  of  messages  were  for 
them  (eating  habits,  exercise,  medication,  reducing  risks,  coping  with  problems).  Messages  about 
medication  were  perceived  as  most  useful  (mean=5.67,  with  no  response  below  the  midpoint  (4). 
Messages  about  exercise  and  coping  with  problems  were  rated  lower  (both  means  =  4.78,  with  responses 
ranging  from  1  to  7. 


The  satisfaction  questionnaire  included  several  open-ended  questions  asking  participants  about  what  they 
liked  and  what  they  disliked  about  aspects  of  the  program,  when  they  felt  a  need  for  person-to-person 
contact  and  in  what  ways  the  program  helped  them  to  manage  their  diabetes  and  meet  the  goals  they  had 
established  at  the  outset.  Responses  to  the  open-ended  questions  were  sparse,  especially  in  the  telephone 
group.  In  answering  a  question  about  how  the  class  portion  of  the  program  could  be  improved,  those  in 
the  telephone  group  who  responded  had  no  suggestions  for  improvement,  saying  it  was  informative, 
interesting,  helpful,  as  is.  Those  in  the  computer  group  who  responded  were  more  likely  to  make 
suggestions  for  improvement  including  demonstrations  of  diabetes  websites  and  on-line  tools,  more 
information  about  diet,  and  more  on  natural  products  to  improve  “sugar.”  In  terms  of  what  they  liked 
and  found  helpful  about  the  follow-up,  those  in  the  computer  group  mentioned  the  ease  and  convenience 
(not  set  schedule,  could  do  from  home)  of  the  computer  access,  the  helpfulness  of  the  information  (for 
example,  about  meal  planning),  and.  the  reminders  about  diet,  exercise,  and  medication.  Participants  in 
the  telephone  group  mentioned  the  friendliness  and  helpfulness  of  the  person  on  the  telephone,  and  the 
information  provided.  In  terms  of  things  they  did  not  like  about  the  follow-up  or  would  change,  computer 
group  participants  mentioned  lack  of  personal  contact,  and  the  blandness  or  sameness  of  the  information 
and  questions.  They  suggested  more  content  and  quizzes  to  reinforce  the  information.  Telephone  group 
participants  also  mentioned  the  lack  of  personal  contact,  and  the  lack  of  helpful  or  additional  information. 
They  suggested  more  frequent  phone  calls  and  occasional  in-person  meetings. 


77 


Discussion  and  Conclusions 


This  study  was  conducted  over  a  nine-month  time  period.  The  results  demonstrated  an  overall  diabetes 
knowledge  gain  from  baseline  to  the  end  of  the  study  time  period,  but  when  the  telephone  and  computer 
groups  were  looked  at  individually,  only  the  telephone  group’s  knowledge  scores  showed  a  statistically 
significant  gain.  There  were  no  statistically  significant  differences  between  the  groups  in  behavioral 
(clinical)  measures.  The  A  1C  readings  dropped  significant  in  the  first  3  months  following  the  classes, 
suggesting  that  the  educational  classes  may  have  had  some  impact  on  the  participants’  dietary  behaviors, 
but  there  was  no  evidence  that  the  follow-up  messages  enhanced  the  impact.  Although  the  computer 
group’s  HDL  went  up  significantly,  it  was  accompanied  by  a  significant  increase  in  LDL.  There  were  no 
significant  differences  between  the  two  groups  in  participants’  satisfaction  ratings  at  the  end  of  the  study, 
but  there  was  a  tendency  for  the  computer  group’s  mean  responses  to  items  about  the  program  to  be  equal 
to  or  higher  than  the  telephone  group’s  mean,  providing  a  suggestion  that  those  who  received  the 
computer  group  were  more  satisfied  than  those  who  received  the  telephone  follow-up. 


Although  the  satisfaction  results  gave  a  hint  that  those  who  received  the  computer  intervention  were  more 
satisfied  than  those  who  received  the  telephone  follow-up,  there  was  no  evidence  that  this  higher  level  of 
satisfaction  translated  into  stronger  increases  in  knowledge  or  improvements  in  behavioral  (clinical) 
measures.  Nor,  except  for  the  short-term  drop  in  A  1C,  did  the  increased  knowledge  in  the  telephone 
group  result  in  improvements  in  behavioral  measures.  The  participants  n  the  computer  group  reported 
being  comfortable  with  the  computer  medium,  but  the  call  for  personal  interaction  suggests  the  need  for  a 
more  interactive  interface,  and  possibly  one  that  is  tailored  to  the  individual’s  needs  and  concerns.  For 
example,  if  the  participant  expressed  greatest  concern  about  exercise  in  a  particular  session,  the  computer 
messages  would  focus  on  exercise  for  that  session,  thereby  making  the  user  feel  that  the  computer  was 
responding  to  his  or  her  concerns. 

In  the  system  that  was  implemented,  the  individual  responded  to  questions  at  each  follow-up  session,  but 
received  no  acknowledgement  of  or  feedback  about  their  responses.  An  interactive  system  may  engage 
the  participants  more  strongly.  Data  indicating  that  dieters  who  keep  track  of  what  they  eat  each  day  are 
more  likely  to  lose  weight  [heard  in  presentation;  need  to  find  reference]  than  those  who  do  not  suggests 
that  the  computer  interface  should  require  people  focused  on  diet  to  record  what  they  had  eaten.  Similarly 
those  focused  on  exercise,  could  record  their  specific  exercise  activities. 

Glycemic  control  is  a  complex  process  involving  dynamic  management  of  diet,  exercise,  and  medication 
and  individuals  with  diabetes  can  experience  considerable  difficulty  in  applying  the  knowledge  they  have 
to  their  own  (Klein  et  al.,  2007).  Reinforcement  of  knowledge  gained  in  classes  and  follow-up  support  to 
apply  this  knowledge  to  an  individual’s  day  to  day  situations  seems  to  require  a  more  tailored  and 
dynamically  responsive  process  that,  at  least  for  a  while,  until  habits  are  established  and  the  knowledge 
has  been  applied  in  multiple  situations,  may  need  to  be  offered  daily,  rather  than  weekly  or  bimonthly. 
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3  Conenidugh  Diabetes  Instil ure  Welcome  Microsoft  Internet  Explorer 
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Welcome 


W  her?  to  more  on 


Welcome  to  the  Conemaugh  Diabetes  Institute  educational  portal-  we’re  glad  v^ou're  joining  usl  Over  the 
next  nine  months  this  portal  will  provide  you  with  valuable  information  to  help  you  manage  your  diabetes 
We'll  also  ask  you  a  few  quick  questions  to  help  you  see  how  well  you’re  doing.  These  'educational 
episodes’  will  be  available  every  other  week,  and  you'll  get  an  email  letting  you  know  when  a  new  one  is 
available.  We  hope  you  enjoy  each  episode  and  find  them  valuable! 
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Screen  shot  of  the  computer  follow-up  group  email  that  the  participant  received  bimonthly. 
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Episode  2  -  Being  Active  GaSack  Click  here  to  move  t 


TIPi  Exercise  has  the  same  effect  on  blood  sugar  as  insulin.  !t  lowers  blood  sugar, 
making  you  more  sensitive  to  insulin, 


The  goat  for  exercise  is  150  minutes  per  week  This  is  equal  to  30  minutes  5  days  a 
week  If  you  are  Just  starting  to  exercise,  start  slow.  Maybe.  10  minutes  3  times  a  week. 
Walking  is  a  great  exercise.  However,  you  could  also  do  chair  exercises,  water  exercises, 
or  lightweight  lifting  tf  you  are  having  a  rough  time  deciding  whet  exercise  is  best  for  you, 
please  consult  your  doctor. 


Ga  Bach  Click  here  to  move  on 


Screen  shot  of  one  of  the  educational  modules(above)  and  questions  (below)  the  computer  group 
received. 
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Episode  3  -  Taking 
Medications 


gq  Back  Click  here  in  move,  on 


Have  you  added  any  new  medications  since  the  last  time  you  visited  this  site? 

□  Yes 

□  NO 

Have  you  stopped  taking  any  medications  since  the  last  time  you  visited  this  site? 

□  Yes 

□  NO 


Have  there  been  any  changes  to  your  medications’  dosages  since  the  last  time  you 
visited  this  site? 

□  Yes 

□  NO 
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Have  there  been  any  changes  to  your  medications*  frequency  since  the  last  time  you 
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DSME  Follow-up  Study 


Episode  Schedule  for  computer  group 


Episode  1  (Week  1) 

Topic 

Healthy  Eating 

Nutritional  Tip:  Milk  and  yogurt  are  good  sources  of  calcium  and  protein  check  the  nutritional  facts  on 
the  label 

Educational  Content 

Read  labels  for  serving  size,  servings  per  container,  and  total  carbohydrates  to  determine  the  total  amount 
of  carbohydrates  you  will  consume.  Remember,  15  grams  of  carbohydrates  equal  “1  carb  serving”. 

Questions 


Episode  2  (Week  2) 

Topic 

Healthy  Eating 

Nutritional  Tip:  A  one  cup  serving  of  broccoli  is  the  size  of  a  tennis  ball. 

Educational  Content 

Are  you  eating  the  proper  portion  sizes?  If  you  are  not  sure,  weigh  and  measure  your  food.  Here  are 
some  serving  size  guidelines: 

•  Thumb  =  one  tablespoon 

•  Deck  of  cards  ~  3  oz  portion  of  meat  or  fish 

•  Tennis  ball  =  1  cup 

•  Using  smaller  plates  will  help  you  eat  less. 


Questions 


Episode  3  (Week  3) 

Topic 

Healthy  Eating 
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Nutritional  Tip:  Fresh  and  frozen  vegetables  have  less  added  salt  than  canned  vegetables.  Drain  and  rinse 
the  vegetables  if  you  want  to  remove  the  salt. 

Educational  Content 

Don’t  forget  to  eat  your  fruits  and  vegetables.  The  ADA  Food  Pyramid  Guide  recommends  5  to  9 
servings  of  fruits  and  vegetables  daily.  The  more  color  in  your  selection  of  fruits  and  vegetables,  the 
more  nutrients  you  will  get  from  the  food. 

Questions 


Episode  4  (Week  4) 

Topic 

Healthy  Eating 

Nutritional  Tip:  One  serving  of  alcohol  is  approximately  80-150  calories,  one  16  ounce  chocolate 
milkshake  at  McDonalds  has  580  calories  and  102  grams  of  carbohydrate,  one  Minute  Maid  apple  juice 
box  has  90  calories  and  23  carbs.  One  16  ounce  coke  has  150  calories  and  40  carbs,  one  small  diet  coke 
has  NO  calories  and  NO  carbs 

Educational  Content 


Some  drinks,  especially  alcohol,  contain  carbohydrates  and  need  to  be  worked  into  your  meal  plan. 
Water  is  the  cheapest  drink  and  is  healthy  for  you.  Generally,  each  person  should  drink  between  6  to  8 
glasses  of  liquid  daily,  preferably  water. 

Questions 


Episode  5  (Week  6) 

Topic 

Healthy  Eating 

Nutritional  Tip:  All  fats  are  high  in  calories,  limit  serving  sizes  for  good  nutrition  and  health. 


Educational  Content 


For  each  gram  of  fat  you  eat  you  get  twice  the  calories  as  you  do  from  protein  or  carbohydrates.  One 
gram  of  protein  equals  4  calories;  one  gram  of  carbohydrate  equals  4  calories;  one  gram  of  fat  equals  9 
calories. 

Questions 


Episode  6  (Week  8) 
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Topic 

Healthy  Eating 

Nutritional  Tip:  Dietary  fats  are  a  part  of  a  healthy  diet,  the  type  of  fat  eaten  is  important  for  heart  health. 
Choose  most  of  your  fats  from  plant  sources  like  nuts,  seeds,  vegetable  oils  and  avocados. 

Educational  Content 


A  fat  is  a  fat  is  a  fat  -  when  calories  are  concerned!  As  a  general  rule,  saturated  fat  is  hard  at  room 
temperature  and  raises  cholesterol  levels.  Mono-unsaturated  fat  is  liquid  and  does  not  raise  cholesterol 
levels.  Mono-unsaturated  fats  are  better  for  your  heart.  Choose  oils  such  as  olive  and  canola  when 
cooking. 

Questions 


Episode  7  (Week  10) 

Topic 

Healthy  Eating 

Nutritional  Tip:  Eating  at  least  three  meals  a  day  helps  you  balance  the  amount  of  carbohydrate  you  eat 
throughout  the  day  to  manage  your  blood  glucose.  Skipping  meals  can  lead  to  hypoglycemia  (low  blood 
glucose),  especially  if  you  take  insulin. 

Educational  Content 


It  is  important  not  to  skip  any  meals,  eat  your  meals  on  a  regular  basis,  and  sometimes  snacks  are  needed 
throughout  the  day  and  evening.  You  should  eat  something  every  4  to  5  hours.  By  doing  this,  you 
should  have  better  control  of  your  blood  sugar  levels.  Keep  in  mind  that  what  you  eat  throughout  the  day 
should  not  go  over  the  total  carbs  you  are  allowed. 

Questions 

Episode  8  (Week  12) 

Topic 

Healthy  Eating 

Nutritional  Tip:  Dietary  Recommendations  are  that 
55-60%  of  total  calories  should  come  from  carbohy  drates 
10-20%  of  total  calories  should  come  from  protein 
Less  than  30%  should  come  from  fat 
Educational  Content 


When  you  sit  down  for  a  meal,  draw  an  imaginary  line  through  the  center  of  your  plate.  Draw  a  line  to 
divide  one  section  into  two. 
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•  About  one-fourth  of  your  plate  should  be  filled  with  grains  or  starchy  foods  such  as  rice,  pasta, 
potatoes,  com,  or  peas. 

•  Another  fourth  should  be  protein  -  foods  like  meat,  fish,  poultry,  or  tofu. 

•  For  the  last  half  of  your  plate,  fill  with  non-starchy  vegetables  like  broccoli,  carrots,  cucumbers, 
salad,  tomatoes,  and  cauliflower. 

•  Add  a  glass  of  non-fat  milk. 

•  Finally,  add  a  small  roll  or  piece  of  fruit. 


Questions 


Episode  9  fWeek  14) 

Topic 

Being  Active 

Exercise  Tip:  Exercise  has  the  same  effect  on  blood  sugar  as  insulin.  It  lowers  blood  sugar,  making  you 
more  sensitive  to  insulin. 

Educational  Content 

The  goal  for  exercise  is  150  minutes  per  week.  This  is  equal  to  30  minutes  5  days  a  week.  If  you  are  just 
starting  to  exercise,  start  slow.  Maybe,  10  minutes  3  times  a  week.  Walking  is  a  great  exercise. 
However,  you  could  also  do  chair  exercises,  water  exercises,  or  light  weight  lifting.  If  you  are  having  a 
rough  time  deciding  what  exercise  is  best  for  you,  please  consult  your  doctor. 

Questions 

Episode  10  (Week  16) 

Topic 

Being  Active 

Exercise  Tip:  A  person  weighing  140  pounds  would  burn  6.4  calories/minute  cycling,  8.6  calories/minute 
dancing,  9  calories  /minute  swimming,  and  7.6  calories/minute  walking 

Educational  Content 


Did  you  take  a  walk  today?  Did  you  get  on  your  bike  or  treadmill  today?  Did  you  do  some  form  of 
exercise  today?  Regular  exercise  will  help  you  to  better  control  your  blood  sugar  numbers,  reduce  stress, 
and  make  you  feel  better. 

Questions 


Episode  11  (Week  18) 

Topic 

Being  Active 
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Exercise  Tip:  Any  exercise  is  better  than  no  activity  so  add  more  daily  activity. 
Educational  Content 


How  active  have  you  been  today? 

•  Did  you  take  the  stairs  instead  of  the  elevators? 

•  Did  you  park  further  away  from  the  entrance  to  the  store? 

•  Did  you  stand  and  walk  in  place  during  the  commercial  break? 

•  Did  you  lift  light  weights  during  the  commercial  break? 

•  Did  you  pace  the  kitchen  while  talking  on  the  phone? 

Questions 


Episode  12  (Week  20) 

Topic 

Being  Active 


Exercise  Tip:  To  maintain  your  exercise  program  find  a  buddy,  do  something  you  enjoy,  join  an  exercise 
class,  walk  your  dog. 

Educational  Content 

Exercising  on  a  regular  basis  can  help: 

•  relieve  stress 

•  you  sleep  better 

•  increase  your  energy 

•  decrease  your  blood  pressure 

•  decrease  your  LDL  (lousy  or  bad  which  you  want  low)  cholesterol 

•  increase  your  HDL  (healthy  or  good  which  you  want  high)  cholesterol 

•  strengthen  bones 

•  promote  regularity 

•  improve  strength,  balance,  and  flexibility. 


Questions 


Episode  13  (Week  22) 

Topic 

Monitoring 
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Monitoring  Tip:  Research  has  shown  that  keeping  your  blood  glucose  as  close  to  normal  as  possible,  can 
prevent  or  slow  the  complications  of  diabetes. 

Educational  Content 

Check  your  blood  sugar  on  a  regular  basis  using  different  times  of  the  day.  Your  blood  sugar  target  range 
should  be  between  80  and  130.  If  you  are  consistently  running  above  those  numbers,  consult  your 
physician.  If  you  stay  within  the  target  range,  it  has  been  shown  that  you  may  delay  or  prevent  the  onset 
of  complications. 

Questions 


Episode  14  (Week  24) 

Topic 

Taking  Medications 


Medication  Tip:  ALWAYS  carry  a  list  of  your  medications  and  their  doses  with  you.  The  list  should 
include  your  prescription  meds,  vitamins,  over  the  counter  meds  and  herbal  supplements. 

Also  include  your  medication  allergies. 

Educational  Content 

Make  sure  to  take  your  oral  medications  as  prescribed  by  your  physician,  including  taking  them  at  the 
same  time  every  day.  Short  acting  and  intermediate  acting  insulin,  for  best  results,  should  be  taken  30 
minutes  before  you  eat  your  meal.  Fast  acting  insulin  (including  inhaled  insulin)  should  be  taken 
approximately  10  to  15  minutes  before  you  eat. 

Questions 


Episode  15  fWeek  26) 

Topic 

Problem  Solving 

Sick  Day  Tip:  When  you  are  sick,  your  body  will  release  hormones  that  work  to  help  your  body  Fight 
against  your  illness,  but  they  will  also  make  your  blood  sugar  rise. 

Educational  Content 

When  you  are  sick: 

•  take  your  diabetes  medication  or  insulin 

•  check  your  blood  sugar  every  2  to  4  hours 

•  drink  8  ounces  of  water  or  sugar  free  liquids  every  hour 

•  eat  10  to  15  grams  of  carbohydrates  every  1  to  2  hours 
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•  rest. 


Questions 


Episode  16  fWeek  28) 

Topic 

Healthy  Coping. 


Coping  Tip:  Millions  of  people  have  diabetes.  Although  diabetes  is  a  serious  health  problem,  with  proper 
care  you  can  learn  to  manage  your  diabetes  and  lead  a  full  and  active  life. 

Educational  Content 

Diabetes  is  a  life  long  disease,  it  does  not  go  away.  You  can  live  a  healthy,  productive  life.  Support  may 
be  found  from  friends,  relatives,  and  health  care  providers.  Many  celebrities  live  with  diabetes  including: 
Mary  Tyler  Moore,  Patti  LaBelle,  and  B.  B.  King.  You  can  even  climb  to  the  top  of  Mount  Everest  as 
Will  Cross  (diagnosed  with  Type  1  diabetes)  did  in  2005. 

Questions 


Episode  17  (Week  30) 

Topic 

Reducing  Risks 

Foot  Care  Tips:  High  blood  glucose  levels  increase  the  risk  of  skin  and  foot  infections. 

Never  go  barefoot,  wear  comfortable  shoes  and  socks  that  fit  well,  file  your  toenails  straight  across  with 
an  emery  board,  call  your  health  care  provider  if  you  injure  your  feet  in  any  way  and  use  skin  lotion  on 
the  outside  of  your  feet  to  help  prevent  dryness. 

Educational  Content 

Inspect  your  feet  daily  or  at  least  one  to  two  times  per  week.  It  is  important  to  watch  for  small  cuts,  sores, 
bruises,  etc.  before  they  develop  into  a  larger  problem.  Most  amputations  can  be  prevented  if  problems 
are  caught  early  and  treated. 

Questions 


Episode  18  (Week  32) 

Topic 
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Reducing  Risks 

Risk  Reduction  Tip:  Diabetes  can  cause  damage  to  the  blood  vessels  that  supply  blood  to  the  retina. 
Retinopathy  can  lead  to  blindness. 

Educational  Content 


Have  you  had  your  yearly  eye  exam?  If  not,  make  an  appointment  to  have  one  done.  It  is  important  for 
detecting  and/or  preventing  eye  disease.  Retinopathy  (damage  to  the  retina  of  the  eye)  is  one  of  the 
complications  of  diabetes  and  can  only  be  detected  by  a  dilated  eye  exam.  Call  your  healthcare  provider 
right  away  if  you  notice  any  changes  in  your  sight  such  as:  blurry  vision  or  specks  floating  before  your 
eyes 

Questions 


Episode  19  (Week  34) 

Topic 

Reducing  Risks 

Risk  Reduction  Tip:  Heart  disease  and  stroke  are  much  more  likely  to  occur  in  people  with  diabetes. 
Eating  foods  high  in  saturated  fat,  trans  fat,  and  cholesterol  can  lead  to  excess  lipids  (such  as  cholesterol 
and  triglycerides)  in  your  blood.  Over  time  lipids  can  damage  your  blood  vessels  causing  circulatory 
problems. 

Educational  Content 


ADA  recommends  a  yearly  lipid  profile.  This  includes  total  cholesterol,  triglycerides,  LDL,  and  HDL. 
You  should  “know  your  numbers'*.  Total  cholesterol  should  be  less  than  200;  triglycerides  less  than  150; 
LDL  less  than  1 00;  and  HDL  greater  than  40  for  men  and  greater  than  50  for  women. 

Questions 


Episode  20  (Week  36) 

Topic 

Reducing  Risks 

Risk  Reduction  Tip:  The  A1C  test  measures  your  average  blood  glucose  level  over  the  past  2  or  3  months. 
The  results  of  this  test,  which  is  done  by  your  health  care  provider,  combined  with  your  own  blood 
glucose  testing  will  help  assess  your  risk  of  diabetes  complications 

Educational  Content 

ADA  recommends  an  A1C  blood  test  at  least  twice  a  year.  You  should  “know  your  number1'  and  it 
should  be  less  than  7.0 

Questions 
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Episode  21  (Week  38) 

Topic 

Reducing  Risks 


Risk  Reduction  Tip:  Cigarette  smoking  is  the  most  important  modifiable  cause  of  premature  death  for 
those  with  and  without  diabetes.  Smoking  is  also  related  to  the  earlier  development  of  microvascular 
complications 

Educational  Content 

Do  you  smoke?  If  you  do,  you  should  consider  stopping.  Smoking  puts  people  with  diabetes  at  greater 
risk  for  developing  cardio-vascular  complications  which  include  heart  attacks  and  strokes.  Contact  your 
local  health  care  facility  to  see  if  they  have  smoking  cessation  classes. 

Questions 


DSME  follow-up,  clinical,  and  medication  data  questions 


One  or  more  of  the  following  questions  may  be  asked  to  the  patient  during  the  course  of  an  episode . 


Have  you  added  any  new  medications  since  the  last  time  you  visited  this  site? 

•  Yes 

•  No 

Have  you  stopped  taking  any  medications  since  the  last  time  you  visited  this  site? 

•  Yes 

•  No 

Have  there  been  any  changes  to  your  medications1  dosages  since  the  last  time  you  visited  this  site? 

•  Yes 

•  No 

Have  there  been  any  changes  to  your  medications1  frequency  since  the  last  time  you  visited  this  site? 

•  Yes 

•  No 
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How  well  have  you  been  following  your  meal  plan? 


•  Not  at  all 

•  Occasionally 

•  Most  of  the  time 


How  often  are  you  testing  your  blood  sugar? 

•  Once  a  day 

•  Twice  a  day 

•  Other 

What  are  your  average  blood  sugar  results  over  the  last  month?  Most  of  the  time: 

•  Under  140 

•  Between  140  -  160 

•  Between  160  -  200 

•  Over  200 

Do  you  exercise  on  a  regular  exercise? 

•  Yes 

•  No 


How  often  do  you  see  your  family  physician? 

•  More  than  once  a  year 

•  Once  a  year 

•  Only  when  I'm  sick  (less  than  once  a  year) 

•  Never 


When  was  your  last  appointment  with  your  family  physician? 

•  Last  6  months 

•  Last  year 

•  Over  a  year  ago 

•  Never 

•  Don’t  know 


When  is  your  next  scheduled  appointment  with  your  family  physician? 

•  In  the  next  6  months 

•  In  the  next  year 

•  I  don't  have  one  scheduled 
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When  was  your  last  dilated  eye  exam? 


•  Last  6  months  Good  job;  you  need  to  maintain  good  eye  health. 

•  Last  year  Good  job;  you  need  to  maintain  good  eye  health. 

•  Over  a  year  ago  Yearly  eye  exams  are  recommended;  protect  your  eye  health. 

•  Never  Diabetes  puts  you  at  higher  risk  for  eye  complications. 

•  Don't  know  If  you  can't  remember;  you’re  probably  past  due  for  an  eye  exam. 

Have  you  had  an  A1C  test  since  the  last  time  you  visited  this  site. 

•  Yes 

•  No 

What  was  the  result  of  your  last  A  1C  test? 

•  [nnn] 


Have  you  had  your  urine  checked  for  protein  since  the  last  time  you  visited  this  site. 

•  Yes 

•  No 


What  was  the  result  of  your  last  urine  check  for  protein? 

•  Normal 

•  Abnormal 

•  Don't  know 

Have  you  had  your  cholesterol  checked  since  the  last  time  you  visited  this  site. 

•  Yes 

•  No 


What  was  the  result  of  your  last  cholesterol  check? 

•  Total:  [nnn] 

•  HDL:  [nnn] 

•  LDL:  [nnn] 


During  your  educational  session  you  chose  the  following  behavior  goal: 
[GOAL] 
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How  well  have  you  been  meeting  this  goal: 


•  Never  It  may  be  helpful  to  review  the  written  material  from  >  our  diabetes  classes 

•  Rarely  Try  to  identify  what’s  keeping  you  from  meeting  your  goal. 

•  Sometimes  It’s  a  work  in  progress;  don't  give  up. 

•  Mostly  You’re  headed  in  the  right  direction;  keep  going 

•  Always  Wonderful!  Keep  up  the  good  work. 


PRODUCTIVITY  FOR  AWARD  NUMBER  N00014-04-1-0825 


Refereed  Articles  *  =  Data  Based 

Grady,  J.L.,  &  Hobbins,  B.  (2009).  How  can  state  boards  of  nursing  encourage  curricular  reform? 
[Headlines  from  the  NLN].  Nursing  Education  Perspectives,  30(1),  59-61. 


*Neuman,  L.H.,  Pardue,  K.T.,  Grady,  J.L.,  Gray,  M.T.,  Hobbins,  B.,  Edelstein,  J.  (2009  ).  What  does  an 
innovative  teaching  assignment  strategy  mean  to  nursing  students?  Nursing  Education  Perspectives . 
30(3),  159-163. 


Workshops  and  Conferences: 

Published  Abstracts 

Winters,  J.  &  Grady,  J.L.,  (2009).  Using  telehealth  technologies  for  teaching  remote 
assessment  to  undergraduate  nursing  students  [Abstract].  Telemedicine  and  e-Health, 
15,  supplement  1,  63. 
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INVITED: 


2010  Videoconferencing  far  Clinical  Learning  and  Technology  Tools  ta  Enhonce  Clinical 

Learning  (2  sessions).  Emerging  Technologies  in  Nurse  Education  Conference  - 
Sponsored  by  Contemporary  Forums.  Boston,  MA.  (Conference  scheduled  for 
July  19-20,  2010). 


2009  Developing  a  Virtuol  Classroom  .  ELITE  Faculty  Development  Workshop. 

University  of  Pittsburgh  School  of  Nursing,  Pittsburgh  PA. 


Learning  by  Simulation:  State  afthe  Science?  (Keynote)  Indiana  University  of 
Pennsylvania  Department  of  Nursing  and  Allied  Health,  Simulation  Conference, 
Indiana  PA. 


PEER  REVIEWED  PRESENTATIONS 

2010  Reoching  Out  ta  Rural  Elders:  Evaluating  Telehealth  with  a  Human  Touch  in 

Assisted  Living  Facilities.  American  Telemedicine  Association  15th  Annual 
International  Meeting  and  Exposition.  San  Antonio,  TX.  (Conference  scheduled 
for  May  16-18,  2010). 


Do  You  Wiki?  Using  o  Wikispace  far  Callobarative  Scholarly  Work.  National 
League  for  Nursing  Education  Summit  2010.  Las  Vegas,  NV.  (Co-present  with 
members  of  Curriculum  Innovation  Task  Group)  -  (Conference  scheduled  for 
Sept  29-Oct  2,  2010). 


2009  Targeting  Diabetes  in  Adolescence:  Evaluation  af  an  e-Health  Approach  ta 

Schaol-Based  Care .  12th  Annual  Conference  of  the  Canadian  Society  of 
Telehealth.  Vancouver,  BC,  Canada. 
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Introducing  Baccalaureate  Students  to  Telehealth  Nursing .  12th  Annual 
Conference  of  the  Canadian  Society  of  Telehealth.  Vancouver,  BC,  Canada.  Co¬ 
presenter  Jill  Winters,  PhD,  RN. 


Introducing  Baccaluareate  Students  to  Telehealth  Nursing.  American  Association 
of  Colleges  of  Nursing  2009  Baccalaureate  Education  Conference.  Chicago,  IL. 
Co-presenter  Jill  Winters,  PhD,  RN. 


APPOINTMENTS/HONORS 

2010  Received  award  for  "Outstanding  Contributions  to  Telehealth  Nursing,"  Presented 

annually  by  the  Telehealth  Nursing  Special  Interest  Group  of  the  American  Telemedicine 
Association. 

Appointed  member,  Academy  of  Nursing  Education  Review  Panel, 

2009-10. 


Named  to  Pennsylvania  Business  Central  "Top  100  People  of  2009" 


2009  Innovative  Use  of  Store  and  Forward  Technology  Award,  Canadian 

Society  of  Telehealth 


Appointed  member,  Academy  of  Nursing  Education  Review  Panel, 
2008-09. 


Peer  review  panel,  Nursing  Education  Perspectives 


Selected  as  reviewer.  National  League  for  Nursing  (NLN)  Nursing  Education  Research 
Grant  Program,  2009  cycle. 


Peer  reviewer  for  American  Telemedicine  Association  (ATA)14th  Annual  International 
Meeting  and  Trade  Show.  Las  Vegas,  NV. 
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